You Are Not a Marketplace Founder (Yet): A Stanford Professor on Friction, Data and Experiments

Johari says people think Airbnb sells rooms and Uber sells rides, but the hosts and drivers sell those. The platforms sell the removal of friction, which economists call transaction costs. He points out that markets fail when no one knows who is willing to drive you at 10 AM on a Friday or who will let you sleep in their living room.

This means both sides of a marketplace are customers, since drivers and hosts also depend on the platform to earn money. Johari thinks that forgetting that taking away transaction costs is the value proposition is a fundamental misunderstanding. Ancient markets like the Greek agora or Trajan's Market were made of stone and hard to rearrange, whereas today technology lets operators re-architect a marketplace on the fly.

He describes data science in marketplaces as a cycle of three problems: finding potential matches, making a match, and learning from the matches that happen. Search and recommendation tackle the first, triaging applicants tackles the second, and ratings plus passive signals such as leaving a stay early feed the third. Every marketplace in every vertical, he says, has those three problems and relies on algorithms to solve them.

Asked why some categories, such as cleaners or on-demand tasks, keep failing as marketplaces, Johari says the most common failure mode is thinking too much about being a marketplace before you are one. He notes that he is a professor, not a scaled marketplace entrepreneur, and is cautious about the cleaning industry in particular. His main case is UrbanSitter, which began by solving a friction parents knew: paying a high-school babysitter without cash.

UrbanSitter then used Facebook networks to create trusted introductions and gain liquidity, and later shifted from charging for payments to charging for interviewing and contacting sitters. Johari says you cannot promise to find all the drivers out there if you only have three. At oDesk, the starting value proposition was a trust tool for remote work, with screenshots and tracking so workers and employers could both be assured.

Lenny adds that 90% of a marketplace founder's problems are the same as any startup's. Johari goes further and says that perhaps there should be no category of "marketplace founder", only founders, since almost any business can be affected by online transactions. He notes OpenAI has effectively become a marketplace through its plugins, whether or not it calls itself one.

Johari warns founders not to overcommit their future. At oDesk, taking a fixed fraction of dollars made sense when tools monitored work, but as relationships matured the platform added less value while still taking about 10%. That invites disintermediation, as when a Thumbtack worker who delivered IKEA furniture handed his business card to Johari's family after the first job.

After the merger with Elance to form Upwork, the company had to rethink monetization given that long relationships may leave the platform. Lenny cites Substack as a positive case, where investing in driving subscribers to writers extended the platform's value, with over 80% of his subscribers coming from its network. Johari agrees it is a good story but notes eBay's difficulties when finer-grained fees broke what sellers saw as a long-standing social contract.

Johari's litmus test is whether you have scaled liquidity, many buyers and many sellers, on both sides. If you have neither, he says, just scale one side as any startup would, and listen to general startup advice. If you have one side, you can keep growing it or choose to use it to attract the other, as Uber did by handing out ride coupons at events to subsidize a base of riders and drivers.

He asks founders to "let the ego go" and be humble about the label, since telling investors you will become a platform is nothing new. Lenny raises the DoorDash question of whether to hire workers instead, and Johari invokes Ronald Coase's question of why firms exist if markets are efficient. Curated labor may be the better answer for something like Stitch Fix stylists or physical therapy, where an ongoing relationship matters.

Johari says that where data science helps depends on context. Pricing matters heavily where the platform sets prices, like Uber or DoorDash, and search matters where finding the needle in the haystack is the main friction. His broader point is philosophical: companies often ask data scientists to build a machine-learning model, which means predicting something, but predictions are only used to make decisions.

At oDesk he was first asked to predict which applicant would be hired, which sounded natural for ranking candidates. But a ranking model trained on past hires just picks up patterns in the past. He compares it to a marketing team sending promotions to customers with the highest lifetime value when the real question is how much more they spend because of the promotion.

In his words, prediction is about correlation while decisions are about causation. A chief marketing officer in his executive class answered that he would send promotions to the top lifetime-value customers, which Johari used to push back gently. So his first demand of any data scientist is to keep in mind that the goal is helping the business decide.

He frames this as moving from machine learning to causal inference. Two ranking algorithms should be compared on whether one produces more bookings, not on which better reproduces past choices. Similarly, hiring-match algorithms should be judged by the later quality of matches, such as ratings and rehires, not by recreating the past.

Johari loves experiments but says you cannot experiment your way out of everything. Companies that proclaim they "test everything" still choose what to test and how long to run it, and incentives push toward incremental ideas and long runs because data scientists are judged on wins. He points to a Microsoft paper on A/B testing with fat tails, which argues for trying more risky ideas and running them for less time.

He dislikes the winners-and-losers vocabulary, since experimentation in science is hypothesis-driven and about learning. Badging is his example: badges that seem great often fail because they pull attention from unbadged listings, and dropping the idea misses what that teaches about attention and inventory. Lenny shares that he led Superhost at Airbnb, which a data scientist feared would wreck the ranking and which turned out to have no measurable short-term impact.

Johari says data science is the accumulation of evidence, not one result, and that statistical significance on a single test should not override what you know. Long-term effects such as retaining hosts are hard to measure. He recommends being "quantified" rather than data-driven: put leadership's competing beliefs alongside the experiment result and decide whether the bet is still worth making.

Johari explains that a feature like Superhost mainly redirects attention, so bookings may not rise, given a limited number of Superhosts. In another company, he and a team steered struggling new supply toward experienced counterparts, which then hurt the existing side, and metrics kept swinging. He quotes Servaes Tholen, formerly CFO of Upwork, who said many consequential changes create winners and losers, and the question is whether the winners matter more to the business.

On changing incentives, he says leaders should expect data scientists to share what they learn about the business, not only rigorous results. He suggests Bayesian A/B testing as a technical lever, since encoding prior beliefs rewards experiments that move the prior even when they fail. That creates a positive information externality for future tests.

He tells of a real-estate platform marketing manager who kept an unauthorized holdout group, costing roughly a couple of million dollars but yielding the true value of his team's work. Johari says learning is costly, and teams reward winners in ways that imply testing losers was wasted time. Companies need to see that paying to learn is rational, even though it is not natural.

Asked for the best-designed rating system, Johari says no one has really nailed it and we still use tools from the early eBay and Amazon days. A major problem is rating inflation, documented by colleague John Horton, driven by reciprocity and norming, where a four-star rating starts to feel like punishing someone. One fix is renorming, such as making the top rating mean "exceeded expectations" or asking guests to compare with past stays, which Airbnb did.

He also cautions about averaging: for an established restaurant with 10,000 reviews the next review is irrelevant, but for a newcomer a first negative review can be crushing. Early eBay work found an immediate hit of around 8% to expected revenue and a link to leaving the platform. Blending a prior belief into the average can pull an unlucky newcomer up so they still get a chance.

Lenny describes launching double-blind reviews at Airbnb, which mainly raised the review rate. Johari adds the idea of the "sound of silence": Steve Tadelis and eBay colleagues found that an effective percent positive, which counts ratings never left, predicts seller performance better. He believes rating systems are understudied given that seeing what happened after a match is the biggest change from ancient markets.

Johari rejects the idea that LLMs will automate data science away. Coding and dashboards get faster, but AI massively expands the hypotheses and explanations you can consider, which puts more pressure on humans to narrow down what matters. With 1,000 marketing creatives instead of 10, he asks, what are you looking for and how do you know you found something good enough?

In the lightning round he recommends *How to Lie with Statistics*, David Freedman's writing on "shoe leather" statistics, and *Four Thousand Weeks*. His interview question for candidates is to imagine everything works out and ask who is affected and why it matters. His motto is to slow down and build meaningful mental models, and he would no longer ask coding questions since AI can help with code.

He says what surprises people about Stanford is that no one asks for credentials before a conversation, a culture he links to strength across all schools and the weather. He closes by saying the thing the world needs is data literacy, since AI also generates a lot of prose and extra explanations that can mislead.