Find the Must-Have Users First: Sean Ellis on Product-Market Fit and Sustainable Growth

Ellis describes the test as a simple way to find out whether anyone sees your product as a must-have, and ideally who and how many people do. Users are asked how they would feel if they could no longer use the product, with the options very disappointed, somewhat disappointed, not disappointed, or already stopped using it. The people who answer "very disappointed" are, in his words, a powerful vein to dig into.

Lenny Rachitsky summarizes the rule of thumb that 40% or more saying "very disappointed" signals product-market fit. Ellis calls it a leading indicator rather than proof, since the lagging indicator is whether people actually keep using the product. Retention cohorts are more accurate but take time to mature, whereas the survey works on day one and doesn't need a good analytics system.

The 40% figure was not in the original idea. Ellis first wanted a filter so he wouldn't treat all customer feedback equally and could focus on people who really cared. While working with a couple of YC-backed companies he shared the question with many startups in Silicon Valley and noticed a pattern: products with a high enough share of very disappointed users mostly did well, and those with low scores tended to suffer.

He says it can be asked at any stage, even with a first MVP, because the aim is to learn whether the product resonates with anyone. For him it also protects against a personal risk: taking a growth role at a company where nobody actually cares about the product.

Right after leaving Dropbox, Ellis committed to six months of growth work at Lookout, a mobile security company, and the survey came back at only 7% very disappointed. He worried he might be doing the company a disservice by being on payroll while it chased 40%. Digging into the 7%, he found most of them cared about the antivirus feature, even though the product also offered backup and find-my-lost-phone.

The team made two changes. First, they repositioned the product around antivirus, which acts as a filter so that sign-ups arrive with the right expectations. Second, they streamlined onboarding so the first thing a new user did was set up antivirus and see a message that they were now protected. Ellis sums it up as setting the right expectations and then speed to value.

The next cohort scored 40%, after only two weeks of changes, and six months later the score was 60%. Ellis says the company later reached a billion-dollar valuation, becoming one of the early unicorns, and kept adapting after phone makers built similar features in. He credits the early habit of staying close to users with building the muscle to respond as the market changed.

Ellis notes he came at this as a marketer in an engineering-led company, so it was easier to sell sequencing onboarding around the valued benefit than cutting product features. He adds that moving retention is hard, but it is usually more about onboarding people to the right experience than about tactical retention tricks.

A 40% score, Ellis says, only tells you that you haven't built something people don't care about. Before you can grow, you must understand who considers it a must-have, how they use it, what they used before and what problem it solves. One of his favorite follow-ups is "What is the primary benefit you get?", first asked open-ended to crowdsource benefits.

He then turns the answers into a multiple-choice question, asked of a different group, that forces people to pick one of four distinct benefit statements, and follows with "Why is that benefit important to you?" At Xobni, users said it helped them find things faster in email, and the why behind that was "I'm drowning in email." Testing "drowning in email?" as a hook gave the team the context its customers lived in.

Rachitsky concludes that whether you score 7%, 40% or 60%, the best use of the tool is to study the highly disappointed group. Ellis agrees and says to keep peeling back the onion. The roadmap should double down on what must-have customers value, onboarding should steer new users to that experience, messaging should set expectations and acquisition should target people who have the need.

Asked how firm 40% is, Ellis says it isn't very firm. Its power is giving the team a shared target, a point at which it agrees to start growing aggressively. He says early-stage teams often split, with half feeling years away from being ready and half asking what they are waiting for. He recalls the same debate at LogMeIn in the mid-2000s, about when to step on the gas.

On false positives, Ellis says a high score is a good sign, but what drives it matters. Citing Nir Eyal's *Hooked* and its investment step, he describes surveying webs.com, a fairly commoditized website builder facing newer rivals, and getting about 90% very disappointed, one of the highest scores he had seen. Users had invested heavily in building and beautifying their sites. Eventbrite scored high for similar reasons, so switching costs and utility both feed the score.

On timing, he recommends a random sample of people who have really used the product: more than once, and within the last week or two so they haven't churned yet. That excludes people who only visited the home page, signed up or saw a demo. If you test a new onboarding flow, survey only people who went through it.

Responding to a listener question, Ellis says the test isn't suited to one-off products. He wouldn't ask how you'd feel if you could no longer watch a movie you just saw, or attend a workshop you just attended. For those he uses an NPS question as his filter instead. He thinks it is otherwise widely applicable.

On thresholds, Rachitsky notes that Nubank uses 50% because, as he puts it, Brazilians are very nice. Ellis agrees culture matters, mentions launching companies in Hungary where 30% might be enough, and says that what he really wants to know is whether a product can grow. A strong score is no use if you have no idea how to grow the business.

He then gives the origin story. At Xobni his main customers were senior managers, and he expected a standard satisfaction question to produce lukewarm answers from people who are never satisfied. Flipping it to "how would you feel if you could no longer use this?" promised more honesty, since they would certainly be disappointed to lose something they needed. It worked better than satisfaction questions at Dropbox and every company after.

Ellis adds that in Silicon Valley many people get excited about technology for its own sake, but he is practical: if a product doesn't bring real value, long-term success is unlikely. At Dropbox, he saw early beta users shift over six months from mostly wanting to try cool new technology to mostly trying only things they find useful.

Ellis hasn't seen scores fade once reached, though he has seen companies with strong scores fail, which he treats as an execution problem. He says finding product-market fit is still fairly random and very hard. For sample size, he settled on at least 30 responses, a number he says he picked at random and that others later told him was the accepted minimum.

On whether the fans form a big enough market, he segments respondents by how they use the product. If one use case produces 60% very disappointed and another 35%, but far more people use it the second way, the choice is strategic. He says he prefers the more passionate customer base and works outward from it, because the biggest competitive threat for an innovator is being irrelevant.

On tools, he runs a lot of different ones, having sold a survey business called Qualaroo, and says SurveyMonkey with emailed surveys is fine. What matters is that the survey experience is pleasant for the customer and the data is easy to work with. He advises tracking retention cohorts as you grow, and rethinking if you churn the users who said they'd be very disappointed.

Rachitsky points to Nubank's practice of waiting for a 50% threshold before launching a new product publicly. Ellis says they even apply it at the feature level, asking whether a feature is a must-have and, if not, whether it should exist.

Asked what to do at 10 or 15%, Ellis says he has always taken the product as fixed and asked how to market and grow it, since product changes take long. His rule is to ignore users who say they'd be somewhat disappointed, because they're telling you it's a nice-to-have. Reacting to them risks tweaking onboarding and product for their feedback and making it good for everyone but great for no one.

He singles out Superhuman as a company that used the survey in an interesting way, as described on First Round Capital's blog. They first identified the benefit must-have users focus on. Then, among the somewhat-disappointed users who also care about that same benefit, they asked what the product needs for them to tip into must-have. Ellis thinks this addresses his concern about breaking it for the core fans.

He explains that he published the survey freely with the Kissmetrics team in 2012 on survey.io, and it later moved to PMFsurvey.com. One other question on the template asks what people would use instead. People who would be only somewhat disappointed usually have a commodity use case and an easy alternative, so a must-have has to be both valuable and unique.

Ellis clarifies that when he coined growth hacking he did not mean one-off tricks. He meant scrutinizing everything a company does for its impact on sustainable growth, instead of following the marketing-textbook approach of awareness building that startups can't afford. He says a divisive name can be better than going unnoticed, and he wanted to name an approach used by companies like Facebook, LinkedIn and Twitter.

Growth, for him, is about getting as many right people as possible to the state where they'd be very disappointed to lose the product, and the hardest part sits inside the product team. He builds a flywheel around the must-have value: understand it, pick a metric that counts units of that value delivered, and diagram every way to grow that metric, from onboarding and the aha moment to engagement loops and referral.

His usual sequence starts with activation, which tends to fall between product teams focused on the roadmap and marketing teams focused on bringing people in. Next come engagement, referral and the revenue model, and only when those work does he obsess over channels. He still forms two or three acquisition hypotheses early, since he isn't confident he could invent them under pressure later.

Rachitsky observes that many companies do the opposite, driving growth first and fixing onboarding later. Ellis replies that acquisition is so competitive that without efficient conversion, retention and monetization you can't find scalable, profitable channels. New users are also at highest risk of being lost until they reach the aha moment.

At LogMeIn, a free remote-access product, Ellis couldn't profitably spend more than $10,000 a month on growth. The data showed 95% of sign-ups never ran a single remote-control session, so monetization depended on just 5% of users. After he shared the data, the CEO froze the product roadmap, had everyone in product, engineering and design work on it, and told Ellis to stop searching for new channels.

In three months the signup-to-usage rate improved by a thousand percent, from 5% to 50%. The same channels that had topped out at $10,000 a month then scaled to a million dollars a month with a three-month payback, and 80% of new users came through word of mouth.

He quotes Kettering, "A problem well stated is a problem half solved." On one cheap channel, 90% dropped off at the download step and more than ten A/B tests didn't help. Someone suggested emailing registrants who hadn't downloaded, framed as a note from customer service, and the replies said it seemed too good to be true that it was free.

Offering a choice of a paid trial or the free version, with a check mark by the free one, improved downloads by 300%, because the business model made the free version credible. Ellis says the two big conversion levers are increasing desire and reducing friction. He defines an activation metric qualitatively first, then checks correlation with retention, and prefers an experience that can happen in the first session over something like "used it a hundred times."

Rachitsky suggests there are four main growth engines: sales, SEO, virality or word of mouth, and paid. Ellis says it's rarely just one. At the luggage-storage company Bounce, SEO matters because people search for storage in a city, but many customers also see a sign on the street from one of the 10,000 partners. He distinguishes demand generation, like the signs, from demand harvesting, like Google searches.

At LogMeIn, growth came almost entirely from paid search, because a competitor was spending tens of millions of dollars a month on radio and TV creating demand for a premium-only product. LogMeIn could harvest it with a free alternative. At Dropbox, nobody was searching for cloud storage and backup was expensive, so user-get-user loops made more sense. He gets inspiration for channels by asking customers how they found the product and where they normally find products like it.

Ellis says that in his first five years he leaned entirely on testing and measurement, answering a LogMeIn investor's question about talking to customers with "I don't care what they say, I care what they do." The investor pushed him to talk to a customer daily, and he found his experiments improved. It is hard, he says, to run good experiments without deep context.

On referrals, he describes testing double-sided incentives at Xobni and talking with a friend who had tested them at a previous company, before applying the idea at Dropbox, where an intern built the program. He had held back at LogMeIn for fear of breaking a word-of-mouth engine. His warning is that Dropbox already had an amazing referral rate beforehand, so a program is a great accelerant when something is working but cannot fix a product people don't talk about.

Ellis says freemium works only when the free product is so good that people naturally talk about it, while the premium version is distinct enough to justify upgrading. Companies often worry about the second part and make the free version weak, then wonder why word of mouth is poor. For engagement, he points to the natural usage cycle of a product, such as Airbnb's infrequent use, and to the *Hooked* model as a framework.

As a counterexample, he cites Facebook switching its North Star metric from monthly to daily active users after hitting a billion monthly users. What gets measured gets managed, he says, and the change gave the team an incentive to bring people back daily. He thinks it made the product far more addictive and drew pushback.

To choose a North Star metric, he starts from the must-have value uncovered by the survey and picks a metric that reflects delivering it. It should not be a ratio, should be able to rise over time and should correlate with revenue without being revenue. He prefers units of customer value, such as Amazon's monthly purchases, weekly rides at Uber, nights booked at Airbnb or weekly tickets for Eventbrite.

He recommends making the choice a time-capped group conversation: a team given 30 days takes 30 days, but with the right ingredients and a checklist, 30 minutes can produce a pretty good metric.

Answering a question from Andrew Chen about what has changed, Ellis says that early on, being data-driven in customer acquisition was enough to win. Now most online marketers test heavily, so you must be efficient across converting, retaining and monetizing. That requires marketing, product, sales and customer success to work together, which is why few later-stage companies have replicated the approach.

On ICE, which he invented, he thinks the R in RICE is unnecessary because reach is already part of impact. He says ICE exists so ideas can be sourced across the company and compared systematically, letting you explain to people why their idea wasn't chosen. He suggests AI may eventually make that scoring more informed or replace it, since it could model the probability of outcomes.

He uses ChatGPT to draft answers by prompting "how would Sean Ellis answer this?", then tweaks the draft. He believes AI recommendations could ease ego clashes between product and marketing teams, since dispassionate suggestions are easier to accept. With many experiments running, he adds, the bottleneck usually becomes analysis, where AI should help.

His closing advice, borrowed from data scientist Oleg Yakubenkov, is that it comes down to asking the right question at the right time. In his workshops, people who can't find a good answer usually haven't asked the obvious question, such as asking users why they aren't downloading the software. In the lightning round he adds a life motto: focus on reputation and learning over earnings.