Your power users know your product better than you do

I used to work on an analytics tool for data modelling, where the engineers had actually built an internal querying language. What this did was allow users to create custom queries for their data as it was stored in our system. At some point we began to nice that a small group of users were writing queries in it that were more sophisticated than anything the people who'd built the language had in mind, combining bits of the syntax in ways that weren't documented because nobody had thought they were possible. Essentially they were building it into structures that had the internal team stopping to check whether that was even meant to be doable. The product had more in it than its own makers knew, and these few users were the ones who'd gone and found the extra room.

The handful at the edge

My first instinct, and I'm a little embarrassed by how strong it was, was to file all this under noise. It's a handful of people, not typical of your users and not who you had in mind when you pictured one, and to make it worse they're outliers on exactly the axis that makes them awkward to serve, some of the most technically fluent people touching the product, comfortable in a way most of your users never will be and doing things most of your users would never attempt. It is genuinely tempting to treat them as a rounding error, the strange tail you trim off so you can get back to designing for the middle, and there's even a sensible-sounding case for it, that you shouldn't let a handful of power users drag your whole roadmap around.

The honest reason it's tempting, though, is that the alternative is hard. If you decide these people are worth following, you walk straight into an awkward little design problem, which is that you cannot really build an interface for an output you can't predict. Everything downstream of that query language, the way results got laid out on screen, the charts and the tables, the whole visual layer I was responsible for, had been designed around the outputs we already knew about. The power users were generating outputs we hadn't seen and couldn't fully anticipate, so following them didn't mean adding a feature, it meant designing for a shape that didn't exist yet, which is a stranger and more uncomfortable brief than it sounds.

What they were really showing us

What took me a while to properly understand is that they weren't the tail of the user base, they were the front of it. The effortful, slightly obsessive thing a small and capable group does at the edge of a product is very often the ordinary thing a much larger group will want a year or two on, once someone has smoothed the path and made the capability reachable by people without their patience or their skill. So the power users weren't really asking for a niche feature, they were demonstrating, in advance and without meaning to, a level of capability the product was going to need for a lot more people before long, having got there first and by an unusual route. Which left the only question that actually mattered, whether to read them as a fault to correct or a direction to follow, and I'm glad we mostly landed on the second.

Building for what we couldn't predict

Following them took a month or two in the end, which surprised me at the time, and the shape of the response turned out to matter more than the speed of it. We didn't try to build the specific things the power users happened to be doing that week, partly because we couldn't have kept pace, they'd have moved on by the time anything shipped. What we did instead was make the whole output layer more flexible, more able to take an unpredictable query and render it into something legible without our having hand-designed for that exact case, which is to say we built headroom rather than features, room for outputs we couldn't yet draw. And the quietly satisfying part, the thing that made it worth more than keeping a few brilliant users happy, is that the flexibility we were forced into for the edge made the core product more powerful for everyone. Ordinary users, later on, got outputs they'd never have got if a small number of unreasonable people hadn't shown us the ceiling was higher than we'd assumed.

I think the reason this is easy to miss, and easy to actively resist, is that these users don't feel like a gift while it's happening. They feel like a nuisance, generating support questions nobody can answer and bending the product into configurations you're scrambling to hold up, and the natural move when someone is making your product do things you didn't intend is to gently close the gap by constraining what they can do, which is more or less exactly backwards, or at least it was for us. The people out past the edge of what you built are uncomfortable to have around precisely because they're standing where the rest of your market is headed and hasn't arrived, and most weeks it is genuinely hard to tell the difference between a user who's wrong about your product and one who's just early. I don't have a clean test for telling those two apart, only that you have to stay curious enough about the strange ones to actually go and look before you decide which they are.