Shopify’s Anti-Playbook: Optimize for Churn, Hold Experiments for Years, and Ban KPIs

Archie Abrams, VP of product and head of growth at Shopify, frames the company’s scale with two numbers: Shopify is about 10% of U.S. e-commerce, and it handled roughly $235 billion in GMV in 2023—an economy about the size of Finland’s. Because Shopify is largely behind the scenes, many consumers do not realize they are buying from Shopify-powered stores when they are not on Amazon or Walmart.

The churn conversation starts from Shopify’s mission to increase entrepreneurship on the internet. Abrams says Shopify wants to make it as easy as possible to start an online store, even though most new businesses ultimately fail; the goal is to get as many people as possible to try, because the few breakout merchants—the FIGS of the world—make the whole cohort valuable.

Shopify does charge a subscription, but most revenue comes from payments and other services tied directly to merchant success. In a cohort, many first attempts may not succeed, but the successful merchants make the cohort work; that is why Shopify lowers barriers rather than protecting every subscription.

Lenny contrasts this with most SaaS companies, where subscription revenue makes every customer worth keeping. Abrams says the deeper factor is monetization: Shopify grows with merchants as their GMV grows, so it can tolerate churn at the top of the funnel and still win when a small number of merchants become large.

For growth, Abrams says Shopify looks at a cohort of users acquired in a period, such as a quarter, and then measures how much total GMV those merchants produce over the next one to five years. The metric is not GMV per merchant but the total value of the cohort; if the cohort generates GMV, that translates into revenue, gross profit, and money to reinvest.

GMV is a power-law metric, so a few successful merchants drive the success of each cohort. Lenny compares this to angel investing, where most investments fail but a couple of outcomes make the fund; Abrams agrees that the same power-law logic applies to Shopify’s vintages of merchants.

Because the feedback loop is long, Shopify deliberately sets different parts of the org on different time horizons. Core product builds for the right things for commerce 100 years from now, merchant services works on a medium-term horizon for tools like payments and shipping, and growth owns the end-to-end customer journey with leading indicators and long-term monitoring.

Growth still uses short-term metrics, but Abrams says Shopify tries to instrument experiments so it can revisit them a year or more later and see whether the GMV curve changed. That long-term monitoring informs input metrics and, more importantly, holds teams accountable to whether they actually moved long-term GMV.

Abrams encourages teams to look at their biggest experiment winners one or two years later, because Shopify repeatedly finds that the downstream result differs from the initial call. He says 30–40% of experiments that show an early lift have no incremental lift on long-term GMV; the short-term effect is often more pull-forward than teams realize.

The most common pattern is a short-term lift that fades, but Abrams says experiments sometimes flip the other way. The most interesting cases are when a test uncovers a pocket of merchants who become valuable entrepreneurs but were missed by normal short-term measurement.

One example is monetary friction: trial dynamics, trial length, incentives, app-store credits, or price. The common belief is that reducing monetary friction brings in lower-quality merchants, but Abrams says it can causally change someone’s ability to succeed by giving them more time to try the idea; the experiment may unlock a class of people who would have given up.

His advice for teams that cannot wait years is to instrument the earliest signs of success and follow downstream metrics as deep into the funnel as possible. If an experiment shows short-term lift, it is still reasonable to ship it, but teams should not overestimate the long-term impact; neutral experiments are rarely negative, and some become positive later.

Shopify uses two layers of holdouts. Every change in a quarter holds out 5% of users across the board; for changes that affect only new merchants, Shopify runs a 50-50 split for a few weeks, ships the winner to 100%, and then tracks the cohort that was originally assigned to the experiment.

The experimentation tool pings everyone involved at three, six, and 12 months with updated results, so teams cannot hide from what the experiment really produced over time. Abrams says the core success metric is GMV over time, alongside gross profit and the absolute number of merchants reaching certain GMV thresholds.

When Lenny asks for examples of big wins, Abrams points to monetary friction and to the nuts and bolts of sign-up and onboarding. He says Shopify usually wants to collect more information than most teams think in sign-up, then use it to personalize guidance; nailing those flows has never stopped amazing him because it sets merchants up for long-term success.

Onboarding is a recurring source of growth even though Shopify does not optimize for retention directly. Abrams says the fastest way to improve a retention metric is often to constrict the funnel stage above it, which is why Shopify tries to orient teams toward the total number of people who reach the end of their part of the journey.

Abrams says large orgs naturally break the world into funnel stages, and teams then get seduced by their local conversion rate. But it is almost always easier to increase that rate by making the previous step harder—fewer, higher-intent people—so Shopify pushes teams to optimize the absolute number of people who make it through their stage.

If a team focuses on absolute activation, the best move is often to get more people in the door by reducing friction, even though that hurts the conversion rate. Abrams says teams get nervous when retention or LTV falls, but CAC usually falls by more, so the business may be able to spend more and still get more people through each stage.

The growth org has two big groups: growth R&D, which includes product, design, engineering, and data, and growth marketing, which covers paid acquisition, affiliates, email, content, and SEO. Within growth R&D there are three pillars: growth product for landing pages, sign-up, onboarding, monetization, trial incentives, and the home feed; a platform and tooling pillar for experimentation, communications, business intelligence, and martech; and customer support, which sits in growth to be part of the merchant journey.

At the highest level, growth is measured on total cohort value over three or four years, subject to payback guardrails. Growth marketing channels have LTV/CAC guardrails, growth product teams are measured on total gross profit and incremental cohort value, and experiments are ideally measured over a long period to see the incremental cohort-value lift.

Abrams shares two contrasting long-term experiment examples. Payment-failure notifications produced a big short-term lift but no long-term lift, likely because people who let payments fail were not that dedicated; preconfigured sections and blocks in the online store editor showed no lift in paying merchants but a large impact six months later on selling and GMV, because merchants built better stores that converted visitors and created momentum.

That is why Shopify will ship a neutral experiment if its intuition says it probably helps merchants. Abrams says the culture is “aim heavy”: if the original control is not clearly better, Shopify asks what it would have shipped from a blank slate and picks the version it feels better about.

The broader philosophy comes from Tobi, who frames Shopify as a 100-year company. The principles are to make the best product in the world and make money to do more of it, and the “make money” part is secondary to the long-term product; Abrams says this is why Shopify focuses on getting every business to start with it rather than chasing today’s big brands, many of which may not exist in 30–50 years.

Core product teams do not have KPIs or OKRs; Abrams says Shopify rants against them and that in core the question is whether the team has conviction that this is the right technical foundation for commerce. Data is one input, but teams are not held accountable to a metric over six months; they are judged on whether they shipped the right thing, through data, qualitative feedback, and product sense. The upside is risk-taking and merchant-facing product; the downside is subjective debate and squishiness. Glenn, who heads core product, goes incredibly deep on every release, and Shopify’s GSD system requires an ok2 approval from group leads—Glenn for core, Archie for growth, and the relevant lead for merchant services—with a short video and Figma for every shipped project.

As Shopify moved upmarket, it added a sales motion alongside product-led growth. Abrams says the biggest learning for R&D is that sales scale is very different, so quantitative data is harder to use; growth has had to build more qualitative insight with merchant success and sales about onboarding large customers, import tools, and different use cases.

Shopify initially built two separate funnels—one for sales, one for self-service—but is now creating hybrid journeys where merchants can choose or move between them. That broke old metrics and job definitions, because a merchant who came in through self-service and then moved to sales was not valued through the original channel.

Abrams gives the example of an ad driving someone to self-service who later talks to sales and becomes a sales-driven merchant. Because Shopify had separate LTV models for sales and self-service, the large value of that merchant was not attributed back to the ad campaign, which could lead to reducing investment in a campaign that actually worked. Shopify is rebuilding instrumentation, LTV modeling, attribution, and incrementality testing; Abrams says multi-touch attribution has a place, but incrementality—withholding ads from a control group and measuring causal lift—is the gold standard.

Shopify does not have a CMO. Marketing leaders are embedded in growth, revenue marketing sits near sales, brand sits under Harley, PMM sits with product managers, and Shop marketing sits on the consumer side. Abrams says the benefit is proximity to outcomes and faster movement with less coordination; it works because Tobi and Harley hold the cohesive brand story. The downside is that things sometimes get messy.

Abrams’s experience at Udemy taught him that discounting can be a growth lever when a category is unclear. A high list price signaled quality—around $100, like a college course—while a deep discount, often to $10 or 90% off, made the course feel affordable and created urgency.

The deeper insight was the emotional job: buying a course was progress in an educational journey, even if the person did not finish it. Discounting with urgency let people make that emotional journey through purchase, and because the emotional job could repeat, Udemy could retain users by giving them another reason to come back.

In the lightning round, Abrams recommends Scientific Advertising by Claude Hopkins for first-principles copywriting and The Perfect Mile for perseverance. He watched The Sopranos for the first time, uses an AI music creator with his kids, and lives by the motto “the plan is the plan until it’s not.” He also credits his late father, an entrepreneur in technology, for modeling empathy, curiosity, and kindness.

Abrams says he is not very active on social media but is reachable on LinkedIn, and Shopify is hiring growth marketers, PMs, engineers, data people, and UXers. He highlights that Shopify remains fully remote and is hiring across functions, including growth.