The Long Tail, the World Model, and the 1995 Problem: Inside Waymo's Scaling Phase and Robotics' Missing Moment

Vincent's answer to how the LLM and diffusion wave changed Waymo is that nothing had to be thrown out. The foundation-model revolution, as he describes it, is about building very large teacher models that run in the cloud, ingest everything the fleet has collected plus internet data, and model the driver, the car's behavior and the environment. That teacher is then used to train and distill data into the onboard models, which can start simply by receiving a different mode of supervision. From there, Waymo can evolve toward models with more capacity and expressiveness, because every model in the stack got a better teacher and far more information to work from.

The first thing the large models bring is what Vincent calls world knowledge: the semantic understanding of things the fleet's own data may never have captured. A city's police cars and emergency vehicles look different until you have driven past them, but a multimodal model already knows what a general police car or emergency vehicle looks like, and it can recognize an accident scene from an image even though Waymo's data contains few of them. Models such as Gemini or GPT supply that context from the web, and their scale — pretraining on large amounts of visual and text data — also lifts the reasoning and overall capability of the driver model built on top.

What Vincent deliberately keeps out of the model is the guarantee. Anything involving strict safety contracts or regulatory constraints is expressed explicitly, outside the AI model, so that when the model proposes a driving plan, a separate layer can verify that the plan meets those requirements. That arrangement, he argues, gives a strong framework for guaranteeing that the car behaves reasonably at all times while still letting the model exercise its judgment about the right strategy for driving.

Vincent's path to Waymo was accidental: a car accident last year put him out of commission for a few months, and physical therapy meant taking Waymo everywhere. He describes the product as magical precisely because it had no fancy UI and felt like AI that applied to everybody, and since his DeepMind team had picked up the reins while he was away, he took the chance to do something different. He draws a direct line from robotics, since an autonomous car is a robot with camera inputs and actuation outputs, much like a manipulation robot with cameras, arms and fingers. The big difference is the operational domain: in research he was still chasing nominal behavior, getting a robot to make coffee at all, whereas self-driving already has a nominal system at commercial safety and quality and the challenge is scale.

Vincent argues there are no big blockers left in the classic sense. Waymo doesn't drive in snow today, but largely because that wasn't the most pressing thing to work on, and problems such as fog or highway driving have been knocked down one by one. What dominates now is the long tail: an event a human driver might experience once in a lifetime happens to the fleet every week or every month. The exceptional, weird and difficult becomes a routine occurrence, and that is what puts pressure on scaling.

Waymo attacks that tail with simulation, synthesizing scenarios for events it knows are eventualities but has never observed, and validating models against them. It also mines situations where nothing bad happened but something worse could have, then modifies the scenario to make it harder — turning ordinary drivers into drunk or actively adversarial ones — so the system learns to react to the worst case rather than the observed one.

Asked what technical advance could change the landscape again, Vincent points to reliable, physically realistic world models: a digital twin that renders a scene accurately enough to stand in for the real world. Video prediction models such as Sora or Veo are proto-world-models, but he notes they were built for video first because the stakes were lower, and the field chased photorealism before controllability; the current push toward authoring tools is what is forcing control over geometry, content and style. The gap for self-driving is that these models are weakest exactly where Waymo needs them — on the long tail — and that the hard part is causality. Generating a plausible video from correlations is one thing; making a model respond correctly to a counterfactual input is another, and injecting causality into models is a problem machine learning has struggled with for a long time.

Vincent describes Waymo's research posture as a constant trade-off between pushing the state of the art itself and inheriting work from outside. For many parts of the problem there is leverage in academia and other institutions being interested, but autonomous driving has bespoke problems that nobody else is chasing, so Waymo tries to steer the conversation — the Waymo Open Datasets, he notes, were designed for exactly that and are now a standard for AV research. Today Waymo sits at the forefront, which means it can no longer wait for the community to build the next thing. Entering a new city, meanwhile, turns out to be mostly an evaluation exercise: the models are remarkably portable, differences such as emergency vehicle appearance are things the model already recognizes, and the real work is convincing yourself, regulators and the community that robustness has been validated — plus logistics, hence the Uber partnership, and earning the trust of the local community.

Waymo uses cameras, lidar and radar, and Vincent argues they are remarkably complementary because their strengths and weaknesses are orthogonal, so the diversity lets Waymo check that what the camera sees correlates with what the lidar sees and investigate when they disagree. He frames the industry split as a business-strategy split: companies climbing from L2 driver assistance must keep the sensor suite cheap enough for an individually owned car, while an L4 fleet that earns money per mile can afford more sensorization. Waymo chose the opposite route, possibly over-sensorizing, solving the hard problem first and then using the resulting data to decide which costs to cut — he observes that solving the 10x problem is sometimes easier than solving the 1x problem because you have the right North Star.

The sensor story isn't only about performance; it is also about redundancy, and Vincent doesn't expect the need for redundancy to disappear. He takes the common argument — I drive with my eyes, so cameras should be enough — seriously as a proof point, but says he is increasingly convinced the bar for L4 is not human level but above it. Waymo's safety reports already show fewer collisions and reported injuries than the average driver by a significant margin, and he treats that superhuman performance as a business requirement rather than a bonus; Jacob's reference to the viral clip of a scooter rider falling in front of a Waymo car makes the same point. Whether that bar can be met with a simpler sensor suite is, in his view, something the next few years of data will settle, because the argument can now be made with measurements rather than expectations.

Vincent's favorite milestone to cite is an anniversary: this year marks 30 years since the first transcontinental autonomous drive, which crossed the US at more than 60 mph with over 99% autonomy and convinced people at the time that the problem was nearly wrapped up. It took three decades to reach commercial deployment, which is why he is cautious about milestone predictions. What he will commit to is that both technology validation and user validation are done — the only thing standing in the way is scaling — so the next milestones will be about expansion, including Waymo's first international experiment in Tokyo and its first experience driving on the left side of the road.

Vincent frames robotics with the same tension Jacob raises: enormous recent progress against the cautionary tale of the 1995 drive. In robotics, he says, the field is still chasing the nominal case — a generalist robot that can do whatever you ask — and there is no convincing 1995-ride moment yet, though he wouldn't be surprised if a valid proof point arrives in the next couple of years. The gap is that robots generalize well across visual inputs but poorly across motion: most demos do one thing, perhaps on different colored cups or with randomized objects, without generalizing at the level of skills. A robot optimized for a single use case, cheap and dexterous, could still be a real business, but the general-purpose home robot needs breakthroughs first.

The surprise, Vincent says, was how quickly LLM common sense became actionable on a robot. Knowledge that people never think about — a cup goes on the table, not the floor; a microwave is probably in the kitchen — was unavailable to robotics and AI for a long time, and large language models supplied it. That high-level knowledge could be turned into language-conditioned policies quickly and with high performance, which was the first aha moment: language being fuzzy was fine, because it was precise enough, and making language the backbone of robotics was not crazy after all.

The second realization was that robot actions are just another language — not one expressed in words but one expressed in body actions — so robotics can reuse the machinery built for multimodal and multilingual models and suddenly everything starts to gel. Vincent expects the end state to mirror LLMs: one general, easily retargetable backbone model plus the tools to specialize it, whether through fine-tuning or, ideally, through prompting at test time on the fly, just as instruction tuning makes a general model useful for specific tasks.

He segments the robotics field into hardware-first efforts, which build the most capable humanoid they can and assume the tasks will follow, and software-first efforts, which build general intelligence and trust that it can be retargeted to new platforms. His experience with RT-X left him favoring software-first, largely for data reasons: robotics is still data-bound, and putting a very expensive, wobbly robot in the critical path of data acquisition is a tall order, which limits how far that approach scales. Since manipulation with high degrees of freedom is not solved, he argues the priority right now is optimizing how fast and how much data you can collect, not hill-climbing on a solved problem.

Simulation has been wonderful for locomotion and navigation, Vincent says, where the sim-to-real gap was small enough not to be problematic, but manipulation has been a different story. The cost of simulation isn't buying robots; it's the work of making an environment diverse and representative and tuning the physics until contact and performance are realistic, and in manipulation that cost is very high. His experience is that scaling up physical operations to collect real-world data was the faster path, and he is candid that his team could take that path partly because it was a research organization with the appetite to explore it, while many academic groups have a vested interest in simulation because it is far more accessible.

The bottleneck Vincent keeps returning to is a data flywheel, and he thinks the human-robot interaction community should spend far more time on data acquisition — a pitch he made at the HRI conference. Current strategies include kinesthetic teaching, puppeteering, teleoperation with gloves and synthesizing behaviors in simulation, and which one maximizes throughput is an empirical question. What he would most like to see take off is third-party imitation: learning from watching videos of people doing things, which nobody has cracked and which loops back to world models and the ability to infer causality from observation.

One bottleneck has already been removed by multimodal models. Vincent describes showing a robot that moves a can a picture of Taylor Swift: no robot had ever been taught who she is or asked to act on that entity, but the knowledge lived in the large multimodal model and transferred to the robot. What remains is the motion data — the actuation and the physical skill — and he says the jury is still out on the right way to acquire it.

On whether causality needs a new architecture, Vincent's instinct is that it may just need better data engineering. In language models, causal-sounding reasoning and chain of thought were elicited without being engineered into the architecture, given the right data and the right inductive biases. His hope is that robotics gets there through scale and data curation rather than a theoretical leap.

The question Vincent thinks the field cannot avoid is whether robots can generalize in the space of actions the way they already generalize in perception. He also wonders where the 'robotics is just another language' hypothesis breaks — what the one thing is that will require genuinely new techniques. So far the hypothesis keeps holding: the team assumed motion generation would need its own methods, and it turned out that the same diffusion models used for video generation are state of the art for it, another case of robotics not being as special as expected.

Scaling laws are another place where driving looks like language modeling. At Waymo, Vincent says, the team has been studying scaling laws for large behavior and perception models and sees the same log-linear growth with data and model size, just with different constants. The open question is whether driving hits a limit earlier than large language models do.

Computer vision, by contrast, never had its ChatGPT moment, and Vincent's explanation is that vision is only useful when you act on the world. Without physical actors, vision was limited to observation, and the technology works far better in closed-set problems than in open-set real-world parsing, where performance on academic benchmarks overstates what happens in practice. Robotics, in his view, is the use case that unlocks all of that.

The thing Vincent has changed his mind on is reasoning. Chain of thought began as a funny prompting trick and has reshaped how people think about reasoning and how to get performance from it. He offers a personal example: a science-fiction premise he had sat on for ten years because he couldn't tell whether the physics worked was answered in about five minutes, equations included, by Gemini's deep research. The lesson he draws is not about physics but about accessibility — and about the millions of questions he isn't asking because he hasn't internalized that he can.

Vincent thinks test-time compute will generalize well beyond math and code. The space of problems that are hard to generate but easy to verify is broad, and it maps onto the actor-critic, generative-versus-discriminative pattern that recurs throughout reinforcement learning: you trade the hard problem of producing a plausible answer for the easier problem of checking one. Driving is an example — verifying that a proposed plan satisfies hard constraints is much easier than generating the plan in the first place — so he expects many applications to pick up the technique.

The areas most at stake, in his view, are multi-step problems that require credit attribution, which he sums up as 'RL done right.' He admits a love-hate relationship with reinforcement learning: in early robotics, the community bet heavily on RL learning everything from scratch, largely inspired by AlphaGo, and made less progress than it should have as a result. The paradigm he now expects to stick is to bootstrap with a large model, do a lot of supervised learning, and use RL as the final fine-tuning that turns a model into an expert at a specific kind of reasoning.

For the next 12 to 24 months, the question Vincent most wants answered is where the world-model thrust lands. Controllable video and world generation, including purely generative video games, will teach the field a lot, and he sees a real possibility that today's large multimodal architectures cannot become good world models — in which case another architectural leap is needed. If they can, every computer becomes a generative model of something, and the compute demands of that future are one reason the current investment in compute is so large.

In the quick-fire round, Vincent calls humanoid robotics both over-hyped and under-hyped. The investment isn't justified by current capabilities, but if humanoids work in the next few years it will look entirely justified; the risk is that patience runs out and the field enters a 'humanoid winter' that damages robotics broadly. That is also why he thinks anyone working in robotics today should be working on humanoids too — the field can't afford not to make them work. On timelines, he expects progress on LLM models this year to be greater than last year's, and robotics models to advance more as well.

Asked when self-driving rides will outnumber human drivers in the US, Vincent declines a date and offers a vision instead: a future in which his grandchildren are astonished that people used to drive cars by hand, given the accidents and complexity involved. He would love to see it in his lifetime but doesn't know whether he will, and Jacob jokes that manual driving may end up confined to the rural countryside with no traffic around.

His reflex when a new model appears is to check the LMArena leaderboard — 'metrics, not vibes,' as Jacob puts it — and decide whether it deserves attention. He warns that it is easy to fool yourself, because a model answering a hypothetical question behaves very differently from a model used for a real, functional application. The test he applies is whether it actually helps in his life or his business.

On robots in the home, Vincent counts dishwashers and washing machines as robots that simply don't look like robots, and thinks a mobile manipulator is a long way off. Anything in a house has to justify its square footage, and the bar for safety is severe — if a robot put a nick in his wall, even he, a robotics enthusiast, is certain it would be in the trash within half a second. That is why Roombas succeeded, since they only bump the parts of the wall that are fine to bump, and why he expects fixed workstation-style robots, logistics, last-meter delivery, offices and hospitals — places where someone's job is to repaint the wall — to arrive much sooner.

The application Vincent thinks is most under-discussed is education. The public conversation focuses on cheating and assessment, which misses the bigger point that this is a magical, interactive tool for learning; he describes putting his kid in front of a conversation with an agent and watching them learn a topic quickly, engagingly and memorably. His own Udacity course is long obsolete, and he doubts he is needed now, since the tools would do a better job of explaining things and building the right curriculum. He also flags a favorite non-obvious startup: an AI-designed plant-based cheese company whose blue cheese is indistinguishable from the dairy version and is served in top restaurants — a reminder that AI applied to unexpected domains is where a lot of impact will come from.