从 Google+ 到标签收件箱：怎样用证据而不是意见做产品

Gilad joined Gmail in August 2011, and his first assignment was to connect Gmail with Google+. Facebook was growing fast and people were spending hours on it, which alarmed Google, so the company decided to launch its own social network. Google created a new division, built a strategy around Google+, and tied Gmail, YouTube and search to it.

Years of features, redesigns and iterations followed, and by his account none of it worked: people did not need another social network, did not love it and did not use it. Gmail rolled back its Google+ integration a few years later and the product was shut down in 2019. Gilad says that, in hindsight, Google also missed simpler opportunities, since WhatsApp, not far from Google's headquarters, had hundreds of millions of users and threatened Facebook more than Google did.

He calls this the epitome of opinion-based development: we come up with an idea, believe in it, see early indications that look good, and go all in. He admits to making the same mistake many times as a product manager who pushed his own ideas. At its peak, he says, the Google+ team had about 1,000 people in its own buildings, a division the size of Android or Docs. Neither Google's ad revenue nor Facebook were ultimately affected.

Gilad describes Google's first decade as evidence guided: it focused on customers, generated many ideas, looked at data, launched rough betas and expected people to act on the results. "Fail fast" was part of that DNA, and he believes it would have helped Google+. For Google+ the company set that playbook aside and used what he calls plan and execute.

The tabbed inbox was the reverse: a small idea nobody believed in. The team looked at the underlying problem, which was that many passive users received piles of social and promotional mail and never cleared their inbox. When Gilad pitched his fix, colleagues pointed out that earlier ideas for organizing the inbox had not been used, and asked why his was better.

That question sent his team into user research, a more user-centric goal, and more rigorous tests, first on their own inboxes, then with other Googlers and external testers. They ran usability studies and built data-mining and machine-learning teams to categorize mail. Gilad says about 85 to 88 percent of people love the result, even though most of his colleagues know how to manage an inbox and thought splitting promotions and social was a silly idea.

Looking back, Gilad says the difference was that the team had opinions but little confidence in them, so it let evidence decide. He says every successful product company, including Amazon, Airbnb and Apple, found its own way to balance human judgment with evidence, not to eliminate judgment. Lenny adds, jokingly, that the Promotions tab catches his newsletter.

Lenny asks whether top-down product ideas ever make sense, as in the Steve Jobs iPhone story. Gilad says founders matter a lot in startups and scale-ups and need space to push ideas, but the organization should respond with "Where's your evidence?" He says the popular iPhone story is not the real one: it came from trial and error and several projects, many of which failed, and Jobs was the architect who connected the dots, not the sole creator.

On how to challenge leaders, he notes that Google's open culture lets people tell even Sergey and Larry they are wrong, but that debates based on opinion usually go to the most senior person. With hard data on how Google+ was actually doing, he believes the conversation could have gone better. He is careful to say he was not in every discussion.

For people in less open companies, he suggests running a quiet experiment and bringing the results. He says you usually get one of two reactions: the boss is furious, in which case it may be time to update your resume, or more often pleasantly surprised. He points to Jobs himself, who was against phones and multitouch at first but changed his mind as evidence mounted.

To tell whether you are really evidence guided, Gilad lists telltale signs: unclear, output-based or misaligned goals; missing user-facing metrics; heavy time spent on roadmaps; little experimentation or learning; and a disengaged team whose engineers feel measured only on output.

Gilad presents GIST as four layers: goals, ideas, steps and tasks. Goals define what to achieve, ideas are hypotheses to reach them, steps implement and validate ideas in build-measure-learn loops, and tasks are the Jira and Kanban work. He stresses it is a meta framework drawing on lean startup, design thinking, discovery and growth, and that each chapter of his book covers principles, frameworks and process, with the process the part you must adapt.

When teams say they have goals, he says they often use that layer for planning: what to build, by when, with which resources. Real goals describe the desired end state, whereas many companies have top-level revenue goals plus siloed goals for engineering, design and marketing that pull people in different directions. Evidence-guided companies build overarching goals using models such as the value exchange loop, which delivers value to the market and captures value back.

He distinguishes the North Star metric, which measures value created for users, from the top KPI, such as revenue or profit, which measures what the business captures. His examples are WhatsApp's messages sent, Airbnb's nights booked and weekly active learning users at Amplitude, plus a documents-created metric for a productivity suite. Each can be broken into a metrics tree, and the overlap points to metrics that move everything else.

Gilad says trees help estimate the impact of an idea (an activation-rate gain may barely move the top metric), align teams around two shared metrics, and give teams sub-metrics to own. They can also guide team topology, so organizations arrange teams around goals instead of software structure. Lenny sums it up as a math formula for your North Star that should inform where to invest.

At the ideas layer, Gilad says organizations are flooded with ideas and tend to decide through a battle of opinions, politics or the highest-paid person's opinion. Predicting the best idea is nearly impossible because of uncertainty about users, the market and technology, but cognitive biases convince us ours is superior. He recommends ICE, impact, confidence and ease, to evaluate ideas in a more objective, consistent and transparent way.

He credits Sean Ellis with inventing ICE, including the use of "ease" in place of effort, and prefers it to RICE because he folds reach into impact. Lenny agrees that simple heuristics beat overcomplicated formulas. The trouble Gilad sees is that people guess an impact and then give themselves a high confidence score, such as eight, which undermines the whole system.

To fix this he created the confidence meter, which looks like a thermometer. Opinion-level evidence sits at the bottom: self-conviction is worth 0.01 out of 10, a pitch deck or a link to a hot theme like AI is worth little more, and all of it combined maxes out at 0.1. Reviews by colleagues, back-of-envelope estimates, anecdotal data such as one competitor having the feature, and market data such as surveys move you up a little; medium and high confidence need tests.

Gilad suggests tying investment to confidence: do cheap tests first and invest more as evidence grows, and skip testing when the risk is low, such as reordering a settings page. Lenny notes the meter is also a gentle way to say no, and Gilad has heard that use often: show how other ideas compare on impact and confidence.

Asked about founders who are experienced and short on time, Gilad says the trade-off between speed of delivery and speed of discovery is a false choice. The right metric is not how fast you get bits into production but time to outcomes, getting the right bits there. He argues that evidence-guided work is faster and more resource efficient, since opinion-based work builds the wrong things and learns too late, and good teams build and learn at once.

How much of this to adopt depends on company stage. Early-stage companies do not yet know how they create value, so their goal is product-market fit and heavy OKRs or full metrics trees are overkill. At scale, with many people and much money, you need a systematic way to evaluate ideas.

Gilad has seen two kinds of companies respond most: those emerging into modern product development who struggle to put the pieces together, and those that used to be evidence guided and regressed after a change in management or culture. He advises against transforming everything at once, since fatigue sets in within a quarter. Instead, start where the pain is: goals if there is misalignment, ideas if there are constant debates, steps if you build more than you learn, and tasks if the team is disengaged.

Gilad says organizations often believe they must build an elaborate MVP and then launch it, which is just a beta by another name. His steps model offers a range of cheaper ways to validate an idea's assumptions, in four groups: assessment, fact finding, tests and experiments, and release results. Assessment includes checking alignment with goals, business modeling, ICE analysis, assumption mapping and one-on-one stakeholder conversations.

Fact finding uses data analysis, surveys, competitive analysis, user interviews and field research, and he recommends doing research continuously so you have data to compare ideas against. Tests start by faking it: fake door, smoke, Wizard of Oz, concierge and usability tests. For the tabbed inbox, researchers showed users an HTML facade where, with their permission, the subjects and senders of their top 50 messages were sorted into place by hand, and there was not a line of code behind it.

Mid-level tests are rough but usable versions: early adopter programs, alphas, longitudinal studies and "fish food," testing on your own team. Dogfooding, previews, betas and labs come next, followed by true experiments with control groups such as A/B and multivariate tests, and finally staged releases, percent launches and holdbacks. Gilad's point is that you do not have to start on the expensive right-hand side, so you can poke at many ideas quickly and invest in the good ones.

On tasks, Gilad describes two disconnected worlds: planners who set strategy and roadmaps, and developers in Agile moving tickets, with a PM stuck in the middle feeding a perfectly prioritized backlog. He says such PMs are tired and have no time for research or testing. His proposal is to let developers discover as well as develop, using the GIST board.

The board is per team and shows no more than four key results as goals, the ideas being pursued with their ICE scores, and the next validation steps. At the start of the quarter the team leads set goals, review them with managers and stakeholders, copy the key results to the board, and let the team pick ideas to test using ICE. The board changes constantly, the team meets around it at least every other week, and it creates the missing middle layer of discussion about what the team is trying to achieve and how it is doing.

Lenny summarizes the example: goal of cutting average onboarding time from five and a half days to under two, idea of an onboarding wizard, steps of a mockup usability test, a prototype test and an A/B test. Gilad adds that steps are learning milestones, not engineering ones, and that sharing results lets managers discuss a proposed idea in a more nuanced, objective way. A team can even keep a backlog of validation steps in place of a product backlog.

On roadmaps, Gilad says release roadmaps that promise a feature by October compete with evidence-guided work, and suggests outcome roadmaps such as growing usage in India or tackling churn by a given quarter. Ideas with high confidence can enter a roadmap for delivery. Metrics trees, the mission and team missions supply most of the content for OKRs.