30초 요약
Brian Tolkin joined Uber as employee 100 on the ops team, went on to lead uberPOOL and start product operations, and now runs product and design at Opendoor. He argues that product and ops are two engines of one plane, and that the job is to find the one place where technology gives the most leverage. Along the way he covers product reviews, jobs to be done, experimenting with tiny sample sizes and leading under stress.
- 1Tolkin compares product and ops to a twin-turbine jet: the plane can fly briefly on one engine, but it runs best when both product and operations are working together.
- 2Surge pricing was once a manual system, with city GMs setting the hours, geographies and caps within which the algorithm optimized. The reasons were that the concept was new and risky, local teams knew their cities best, and a fully dynamic geospatial pricing system was hard to build.
- 3Driver onboarding went from one-on-one sessions to classrooms to videos, and then the operational system broke at about a thousand drivers a week. That is the moment to hand scaling over to technology and free the ops team for the next problem.
- 4With low-volume transactions, Tolkin says to run a power analysis before you start, and never pretend a one-month test will give an answer it can't. If significance is out of reach, build conviction other ways, or trust your intuition and ship.
- 5Competition from Zillow was stressful, but Opendoor stayed "competition aware, but not competition focused". The real rival is the traditional way of selling a house, and a vertically integrated product is hard for a software-only company to copy.
- 6Much of the product job is finding the kernel of truth in a sea of signals. In early Uber that meant doing dispatch, pricing and payment well and treating everything else as noise.
챕터별 전체 원고 읽기
Starting in ops gives product leaders a feel for how the business really works
Brian Tolkin, now head of product and design at Opendoor, began at Uber on the operations team, before the company had UberX or any shared rides. He says that work taught him how the business actually runs. A city's success depended on daily inputs on the ground, and he was onboarding drivers one by one and answering support tickets himself, with no centralized support team standing between him and customers.
Host Lenny Rachitsky notes that product teams at companies like Airbnb sometimes look down on ops teams as doing one-off work that won't scale. Tolkin answers with a metaphor Uber and Opendoor both used. Product and ops are a twin-turbine jet that can fly on one engine for a while, but flies most efficiently when both run.
Local teams can iterate faster and talk to customers more efficiently, and they hold qualitative insight that no one in headquarters can. A PM in San Francisco can't walk houses in 50 Opendoor markets or learn the safety nuances of every South American city at Uber. What a PM can do is build a strong feedback loop with the people who do understand those places.
Product operations was born to close the loop between San Francisco and the field
Tolkin explains that later in his Uber career a centralized product team, mostly in San Francisco, was shipping features to a globally distributed operations team. The feedback loop in both directions was weak. Uber's answer was a new function, product operations, which reported into operations but sat physically with product teams and worked like a member of them.
He declines credit for inventing the function. He recalls Google having something similar under a different name, and others at Uber had dabbled with the model before him. What he and his colleagues did was formalize it and build the organization around it.
Rachitsky asks about the story that surge pricing began with a city GM emailing drivers. Tolkin says that was before his time, but he confirms that through 2012 and much of 2013 surge was a manual, human-in-the-loop system. GMs set when it could turn on, which areas it covered and its cap, and the algorithm optimized only within those limits. He gives three reasons: it was a powerful, dangerous new idea, local teams knew about events like a baseball game ending, and a fully dynamic geospatial pricing system was hard to build.
Launch-night chaos makes the best stories, from a placeholder name to Chengdu
Tolkin says UberX got its name by accident. Someone on the ops team modeled the product and, with no name yet, used X as a placeholder. The product was approved and launched quickly, and the name stuck. He says it is too expensive to rebrand now.
He then recounts launching uberPOOL in Chengdu, the first Chinese market for the product, planned for 6 a.m. rush hour on a Monday. Over the weekend, data-center testing left the matching algorithm not working, and he slept about 30 minutes while the team worked with colleagues in the US. Everything came together around 5:30 or 6 a.m., and the team ate street-food pancakes at 7:30 that he remembers as the best meal of his life.
A more recent crunch came at Opendoor in March 2020. Because the company physically enters people's homes, it turned off the core business for a few months when COVID hit. It used that time to virtualize the whole process. Tolkin also gives a short description of Opendoor today: a digital platform where sellers enter details about a home and get an all-cash offer, skipping showings and open houses and choosing their own closing date.
Invest tech where it gives leverage, and hand work to technology once ops hits its limit
Asked for lessons on pairing product and ops, Tolkin lists a few. Both functions need mutual respect, and engineering should be intentional about where technology gives leverage. At early Uber that meant dispatch and pricing, and the team said openly that other areas would not get engineering time for now. He adds that the real world has entropy, because drivers cancel, GPS fails and nobody is home, so products need flex and fail-safes.
He also says companies evolve. Uber's engineering focus later widened as the company matured and optimization and expansion mattered more. The tools got better and the internal infrastructure grew, so things that began one way shifted as the business needed.
His example is driver onboarding. It began as a 90-minute to two-hour in-person session for each Uber Black driver, then small classes, then classes of 20 or 30 for UberX, then a video. Credential checking was fine for a handful of drivers but broke at roughly a thousand a week. At that point technology such as OCR on driver's licenses scaled the process, and it freed dozens, perhaps hundreds, of ops people for the next problem.
Rachitsky relays Casey Winters' hot take that operations is a sign of inefficiency to be squeezed out over time. Tolkin broadly agrees but frames it differently. Ops teams that automate one task can move on to the next, such as optimizing UberX, food delivery, higher-capacity vehicles or better feedback on surge toggles. He says there is always another hill to climb.
Product reviews should make the product better rather than feel like a firing squad
Tolkin is a big fan of product reviews, particularly in ops-driven companies, where cadences such as the weekly business review may not leave room to look up at where the product is headed. He sees two goals. One is accountability and informing an audience, and the primary one is helping teams think through a problem. He wants an intellectual conversation about the work, not a firing squad, though some discussions will get intense.
A lot of that comes from how senior people show up. He believes the people closest to a problem have the best context, so a senior voice should probe and offer ideas explicitly as thoughts, not mandates. The team has depth from 40 to 60 hours a week on the problem, while the leader brings breadth after perhaps three hours.
On logistics, Tolkin keeps reviews small, under about ten people, while the document can circulate widely. The artifacts also help new hires, who can read the last 20 reviews to get up to speed. At Opendoor there are two slots a week that any team can sign up for, and leadership adds teams it hasn't seen for a while so the work cycles through quarterly.
Jobs to be done helps Opendoor empathize with customers who sell a home rarely
Tolkin's view is that every framework is one tool in a toolbox, to be dropped if it isn't working. What he values in jobs to be done is that it forces teams to stand in the customer's shoes. At Uber he rode every day, and at Airbnb people use the product several times a year, so they can intuit the job. Americans move about once every seven years, so Opendoor's team can't rely on that.
The framework also pushes teams to look at context outside the product. A home sale can run for weeks or months, with agents, friends and drives around town. At Opendoor jobs to be done sits in the standard product-review template, which covers context, problem, potential solution, risks and premortem, and measurement of success. Reviews are also sorted by stage, since an ideation review looks very different from a pre-ship one.
Tolkin says the template alone does not improve content. The point is cultural: people keep asking whether something is really the job to be done. An early version might say the job is to get an offer from Opendoor, while the broader job may be price discovery. He adds that jobs to be done at Twitter, where friend Kayvon Beykpour pushed it, became dogmatic, and that he would rather pick the right framework for each job.
When A/B tests are underpowered, build conviction in other ways or trust your judgment
Opendoor has far fewer, far larger transactions than Uber's, so testing is harder. Tolkin says the company still runs A/B tests, which he calls the gold standard, and top-of-funnel product tests are easier than operational-process tests lower down. His first rule is to run a power analysis before starting, to see what effect size you can detect and how long the test would take, and then to be honest about whether that is acceptable.
Sometimes a six-month test is worth it, and you start it in June and plan around the result. The mistake is expecting an answer in a month and acting surprised when it isn't significant. If a classic test isn't possible, there are other ways to build conviction. These include talking to more customers, difference-in-differences, sister or twin cities, geo segmentation, running at 80 percent confidence instead of 95, and long-term holdouts.
Opendoor is quite data-driven, Tolkin says, but if there is no way to test, you can't pretend there is. Rate your conviction as low, medium or high. For consequential decisions at low or medium, talk to more customers or check another person's intuition. If you ship on instinct, set up a feedback loop such as support-ticket volume or feature adoption, so you learn whether you were right.
Stay competition aware but not competition focused, and own the whole vertical
Rachitsky raises Zillow, which tried to do what Opendoor does, then partnered with it instead. Tolkin says Zillow has been a fantastic partner. Large online platforms have audience and reach, while Opendoor offers a seller-side transaction solution, so he describes the relationship as symbiotic. Browsing and discovery start on those platforms, and Opendoor offers the transaction services that let sellers actually move. He says the same holds for other platforms of this kind.
On what Zillow may have underestimated, he avoids speaking for them but says the business is complex. It is not a software-only product: you need strong pricing, product, operations, risk discipline and capital markets. Opendoor has been vertically integrated from day one, so it can't deliver without all of those pieces.
Competition was stressful, Tolkin admits, but Opendoor tries to be competition aware and not competition focused. Most people still sell the traditional way, and homes are the largest asset class in the US, so the market isn't saturated. He compares it to Uber versus Lyft, where transportation demand was large enough for both. Rachitsky adds that Stripe's Atlas had a similar experience with an AngelList competitor, and says the real competitor is customers' default behavior.
Calm leadership comes from exposure to stress and from finding the kernel that matters
On staying calm, Tolkin says early Uber felt like a constant fire drill. His intellectual answer is that when leaders reflect stress onto a team, everyone tenses and outcomes get worse. He also relies on the mantra that you are never as good or as bad as you think. He says experience matters most: you have to go through stressful situations to learn that they pass, and he points to sleeping on the floor in China before the uberPOOL launch.
He also recommends learning from other people's stories, such as podcasts, biographies and the Founders Podcast, which show that the path is never linear. Rachitsky asks about the idea that product work is finding the kernel of truth in a sea of ambiguity. Tolkin agrees: ideas arrive from customer support, customers, executives and field visits, and the core job is separating noise from what moves the customer forward, which means saying no to some good ideas.
His example is early Uber, which skipped sophisticated tooling and a growth team. The product at its simplest connects a rider and driver, prices the trip and handles payment, and until that worked, efficient support-ticket handling or customer-acquisition cost didn't matter. His tip is to write every idea down in a backlog, in a sheet or Jira, so people feel heard and you can step back and decide what really matters.
Early failures reveal what matters, and PMs should be matched to the problem
For failure corner, Tolkin describes the first uberPOOL beta in San Francisco. The team planned to match commuters from specific companies along popular corridors, such as the Marina to Google. They quickly saw that liquidity was the only thing that mattered and that a company-based approach would never reach enough of it. The fix was to push liquidity as hard as they could and learn the product's limits, as with the $5-anywhere-in-San-Francisco promotion.
Rachitsky draws the lesson that perfect beta design matters less than getting enough people in the product. He also recalls the ice cream and puppy promotions. Tolkin says those came from local marketing managers and show the value of distributed, early experiments that the platform could support.
Tolkin closes with a thought about hiring. There are technical PMs, PMs who came from ops and PMs who came from design, and the question should be whether a person's skills and context fit the problem at hand. He says you should shape the team the way you shape a roadmap. Rachitsky calls it person-product fit and notes that many companies instead hire smart generalists.
같은 채널의 다른 글
1:10:27EN
1:30:46EN
1:42:04EN
1:21:44EN
1:01:00EN
1:26:22EN