You only study the customers that stayed
The Field Guide · No. 6

I've spent a good part of my career in customer research of one kind or another, and the thing that took me longest to understand is not any particular finding but where all of it comes from. If you were to trace almost anything you know about your customers back far enough it would (more often than not) come from someone who was still around to tell you. Surveys only seem to reach the people who haven't left; or at least that group is more likely to submit responses. Even an angry support ticket is a kind of loyalty, written by someone who cared enough to complain rather than leave. The customer who bought and then drifted off doesn't leave you any of that, and nor does the prospect who sat through the pitch and went with someone else, so the people you'd most want to hear from are the ones who leave almost nothing behind in the places you look when you want to know who your customer is.
The planes that came back
There's a piece of wartime history that people in analytics circles have told half to death, and I've held off using it for that reason, but it fits this too well to skip. In 1943 the American military was losing a lot of bombers over Europe and wanted to armour them better, and since armour is heavy and you can't cover the whole plane, the question was where to put it. They looked at the bombers coming back from raids and mapped the damage, and it clustered on the fuselage and the wings, so the move was to reinforce those. A statistician named Abraham Wald, a refugee working with a group at Columbia the military had leaned on for this sort of problem, looked at the same maps and said, more or less, that they had it backwards. The planes in front of them were only the ones that had made it home. If the returning bombers were hardly ever hit on the engines or the cockpit, that wasn't because those areas didn't get hit, it was because the planes that took hits there were the ones that never came back to be measured. The armour needed to go where the surviving planes came back clean, because a clean patch there meant the planes hit in that spot hadn't come home at all.
The clean patches in your research
The customer research most teams do is a map of the planes that came back. It's careful and often good, and it'll tell you a lot about the people who stayed. You might learn why they like what they like and where they get stuck, all of it useful. What it can't tell you, structurally, is much about the people who are gone because they aren't in the room. The danger here is that their absence doesn't register as a gap, and instead it registers as consensus. When the only people you hear from are the people you're already serving well, the feedback comes back reassuring, a bit of polish here and a feature request there, and you set the roadmap by it and keep making the product better for the population that was never going to leave. On the flip-side; the reasons people churn or never sign in the first place sit outside the dataset, looking from the inside like they aren't there.
The evidence you have to go and get
Which is why the most useful customer feedback is almost always the feedback you have to go out and get on purpose, because it won't arrive on its own. The failed sales call is the clearest case. Someone sat through a demo, weighed you against whatever else they were looking at, and decided no, and inside that decision is a more honest account of your gaps than a dozen happy customers will give you, and yet it's the one that gets marked closed-lost in the CRM and forgotten, because nobody enjoys ringing back the person who turned them down to ask why. A lost contract is the same, more expensive and more instructive, since the customer who was with you and chose to leave knows what was missing and is the person least inclined to walk you through it. This is where it shows, a slow churn barely announces itself, someone opens the product a little less each month until they've gone, so unless you're watching for that shape you won't clock it as a loss, let alone ask what it was telling you.
None of this gets solved by feeling guilty about it, and I'm wary of turning it into a method, because the honest situation is that survivor feedback will always be easier to gather than the other kind. The people still with you are more willing to talk, and going after the ones who left means chasing people who have no reason to give you their time and a few reasons to be cold about it. So the most useful habit I know of is smaller than a method, it's a question you learn to ask of any research before you trust it, which is who isn't in this, who churned or never showed up, and what would they say if they were in the room. You often can't answer it, and that's alright, because asking it changes what you do with the survivor-shaped picture in front of you, since you stop mistaking it for the whole map and start treating it as the half you can see.
Get new field notes delivered
One or two emails a month. Essays like this one, plus the occasional teardown of what we're seeing across customer calls.


