30 秒看懂
On her fourth Cognitive Revolution appearance, Google DeepMind staff research scientist Keerthana Gopalakrishnan walks Nathan Labenz through Gemini Robotics 2 against a summer of viral Robot Olympics clips. Humanoids can now sprint faster than Usain Bolt, she argues, but footspeed is not what limits useful robots: simulation-friendly locomotion is far ahead of contact-rich manipulation, and the field still sits at GPT-2 on few-shot learning and cross-embodiment. The new stack splits embodied reasoning, a whole-body vision-language-action model, and an on-device variant — pick-and-place is nearing deployable reliability, while tasks that punish mistakes still compound into failure.
- 1Viral sprinting clips are real progress, but footspeed is not on the bottleneck list for whether robots are useful — Gopalakrishnan and her friends are productive people who still cannot outrun Bolt.
- 2The more rigid the object, the better simulation worksWhere contact physics is hard to model — cloth, friction, deformation — sim-to-real starts to break, which is why manipulation lags locomotion.
- 3Two years ago grabbing an apple was hard; Gemini Robotics 2 now does whole-body manipulation while squatting for a watering can. A year ago she would have been surprised.
- 4Generality and mastery are nearly orthogonal axesNarrow AI makes the nth task as expensive as the first; a general baseline is the tide that lifts every boat, the path language models already took.
- 5Robotics is still GPT-2: few-shot learning is not yet broad, and cross-embodiment is the variable language models never had. A brain that only works on your robot in your setup is not a generic brain.
- 6She expects Flash, Pro, and on-device tiers for robots. Six-second reasoning is too slow for a line; overnight laundry can wait — so chores might arrive before factory speed, even if commercial sites still go first on safety.
按章节阅读完整文稿
Footspeed is not on the bottleneck list for useful robots
Keerthana Gopalakrishnan is a staff research scientist at Google DeepMind and research lead for Gemini Robotics. This is her fourth annual appearance on Nathan Labenz's *Cognitive Revolution*, a yearly calibration of how close general-purpose robots are to being broadly useful. Labenz opens with the *AI 2027* forecast: humanoids become useful in mid-2027, and by 2028 a "robot economy" could take shape in which robots build more robot factories. The question hanging over the hour is how that sci-fi timeline lines up with what robotics research can actually ship.
She had not been watching China's Robot Olympics until her father asked about the humanoids. Clips of robots sprinting — and catching fire — were flying around, and friends in India who had moved into student politics were forwarding them as proof that China was building machines faster than the fastest humans. From a research standpoint Gopalakrishnan is not surprised, but she grants that the event captured public imagination in a way few robotics benchmarks do. Collages against the DARPA challenge of twenty years ago, when robots could barely walk, open a door, and then fall over, make the progress hard to dismiss.
Labenz presses the awkward follow-up. A robot that outruns Usain Bolt is spectacular, but is footspeed actually useful, or is the Olympics a sideshow? Gopalakrishnan splits the field into people pushing research and people pushing how fast that research can go. She is "a very productive human," as are her friends, "but we don't run faster than Usain Bolt." Running is a public benchmark and may matter for some defense applications. It is not, in her view, the limiting factor on whether robots can help people do useful things in the physical world.
Locomotion is easy to simulate; contact-rich manipulation is not
Robotics splits into locomotion and manipulation, and locomotion is the one that has advanced furthest because simulation can do so much of the work. Models are trained in simulation and transferred; a flat track with modeled ground contact is almost an ideal sim-to-real task. Labenz noticed robots sprinting into a wall and smashing to pieces — they cannot be training that way on hardware, or they would run out of bodies. That smash, for him, was evidence of massive simulation behind the spectacle.
Sim-to-real is the gap that matters: can what you train in simulation be replicated on the real robot? Gopalakrishnan says the tasks we can learn in simulation tend to be the first ones solved, a pattern familiar from digital agents. Locomotion on flat surfaces models contact with the ground. Manipulation cares about when contact happens and how objects respond, which is where current simulators start to break. Cloth folding is hard because fabric has friction. Pick-and-place is more tractable even in simulation, because contact dynamics are somewhat predictable.
Labenz restates the rule: the more rigid the body — a floor, a wooden cube — and the less deformation, the easier simulation becomes. Gopalakrishnan agrees; where the physics is easier to model, simulation gets in. She did not work on the race demos, so she is speculating, but the published locomotion literature points to motion-imitation controllers fine-tuned with reinforcement learning for a specific robot body and the narrow task of running very fast. Scene understanding and Gemini-style reasoning are, she assumes, mostly skipped for need-for-speed.
Robotics is still GPT-2, even as whole-body control arrives a year early
Asked whether she already delegates anything to robots in daily life, Gopalakrishnan gives a researcher's answer: if robots win a task, she moves on to what they cannot yet do. When she started on manipulation, even pick-and-place was hard; foundation models have made that class of work comparatively easy, so the frontier has shifted from gripper robots to humanoids and hands. Gemini Robotics 2's claim is whole-body control and whole-body manipulation with generalization — taking steps, squatting, manipulating objects in closed loop, and reasoning about unseen objects. "If you had told me a year ago we would do this by now, I would be a bit surprised." Two years ago the team was just starting on humanoids and struggling to grab an apple; she posted a video of a robot squatting in different spots to pick up a watering can.
Labenz maps this onto METR's task-length curve: models can do longer tasks over time, but 50% success is fine on a laptop and a disaster if a robot drops something. Gopalakrishnan says a lot of pick-and-place is close to a strong signal and close to deployment. The deeper design choice is orthogonal axes of generalization and mastery. You can build very narrow AI that is excellent at one or two things, but then the cost of the nth task is the same as the first. Building a general baseline makes it much easier to tune toward mastery. Language models followed that path — specialized models first, then GPT and Gemini as broad baselines you climb from — and she thinks the same thesis will hold for robotics.
Recent demos of one-shot and few-shot robot learning look, to her, like a new prompting modality rather than a solved capability. Older models already generalized from a language prompt such as "pick up the object" on something unseen; then came image prompts with a circle around the target; now people prompt with video. That is in-context learning on a spectrum. The live questions are whether the model is copying or generalizing when the scene changes, and how hard a task you can teach this way. Pick-and-place has lots of data. Tying a trash bag after one demonstration is another matter. She still thinks it is fairly early.
Pressed to score robotics on a one-to-six GPT scale, she still says GPT-2. GPT-3, in her mapping, requires few-shot learning that works well across many different tasks. Robotics also remains "very subject to cross-embodiment." If a brain only works on your robot in your setup, is it a generic brain? Language models never had this problem: her phone or his, Mac or Linux, the behavior is similar. A robot brain that is helpless on a new body is not there yet. A lot of work remains to discount the body out of the intelligence.
One brain, any body: a system-two reasoner, a whole-body VLA, and an on-device twin
Gemini Robotics 2 is a suite of three models. Gemini Robotics ER 2 is the "system two" brain: generic embodied reasoning, based on the Gemini Flash line and tuned toward robotics, available via API for image, video, and semantic understanding. Gemini Robotics 2 is the vision-language-action model that turns an intention from a user or an assistant into how the robot actually moves. Gemini Robotics On-Device 2 is a much smaller version of that action model, small enough to fit on the robot's own computer without living in the cloud. The VLA currently does whole-body control, from fingertips to feet, across a range of form factors, but only for trusted testers; DeepMind is building it with Apptronik, Agile Robots, and Boston Dynamics.
Because ER 2 is in the API, Labenz had a coding agent wrap it, expose `grab_at` and `place_at` tools, and drive a 2D browser canvas. Putting a banana on a plate, and recovering when the object was moved, both worked; he found the model remarkably accessible. Timing traces were less comforting: from prompt to tool call could take six seconds, a long wait in the physical world. Gopalakrishnan says it depends on the application. An assembly line cannot loiter; folding laundry at home, she does not care how fast it is, only how well it does the task, and she can wait. She expects Flash and Pro paradigms in robotics, plus an on-device tier for distributed deployments with poor connectivity. She still expects commercial sites first — more controllable, professionals on site, a higher safety bar at home with toddlers — while building fast and slow models in parallel rather than slow first.
ER 2's 128,000-token context is, densely packed, about three minutes of robot memory, depending on tokenization. That is not enough for a long episode of a robot doing many things; larger Pro models will be more useful for offline analysis. For API users who want more, her context-engineering advice is not to keep the whole history as images. Borrowing a fast-and-slow memory idea, she suggests text summaries of what the robot is doing as it goes. Cooking is the analogy: you might want the frame of flipping an egg, but a lot of the narrative is informative as text, which is more compressible. Manage context the way you manage RAM, and let the ER-to-VLA harness talk in interpretable modalities.
ER was the first robotics model to go out via API because it is closest to Gemini itself and more mature; action models are farther on the frontier. Usage on AI Studio blew past what trusted-tester work had shown, and academics used it for benchmarks, which Gopalakrishnan treats as free information about where the model is weak. Boston Dynamics' Spot has been demoed reading instruments and handing out snacks. The ER-to-VLA interface is a research area called steerability: language, pointing at objects, and video prompts, with a contact surface that should get richer over time. On the on-device model, a result she likes: with 200 examples you can learn a lot of different tasks on a lot of different bodies, because the foundation already understands the physical world and how to control many embodiments.
Tasks that allow retries are easy; frying an egg is a compounding-error problem
Asked whether most failures still sit at the VLA rather than the reasoner, Gopalakrishnan says the hard part is split. Cross-embodiment has taken a large leap, but taking a new body to many tasks at high reliability still has no strong precedent. With two models in the loop — one doing the task, one watching for when to switch — a surprising share of failures is orchestration: when is the subtask done, and how do you progress? Errors compound. Sequence ten tasks and the ER success rate *x* multiplies with the VLA success rate *y*. That compounding is the gap between a demo video and a deployment.
So if you ask a robot to fry an egg, how far does it get? It depends on whether you have data. A specialized algorithm can make that one task reliable. A generic model walked into a home it has never seen is a different story. Visual generalization has improved a lot: lighting and furniture layouts no longer throw the models the way they used to. What still bites is task semantics and the room for error. You can overcook an egg and nobody wants it. The window is narrow.
The distinction she offers is clean: tasks that allow retries are much easier. LEGO assembly can be undone. Crack an egg on the floor and you have a mess — and someone might slip in it. On exotic bodies from the trusted-tester program, she has not seen many; the weirdest is people putting arms on drones. If you have data, she believes you can fit very weird embodiments, but the distribution is ordinary: bimanual arms, hands, humanoids, with everything strange in a long tail.
Hands took the dexterity frontier, and safety is a capability, not a tax
A year and a quarter earlier she had told Labenz that what she most needed from hardware makers was good hands. Gemini Robotics 1 showed dexterity with grippers; Gemini Robotics 2 shows multi-finger dexterity, including tying trash bags and fine control. Her internal model has updated: everything gripper robots could do at the dexterity frontier, hand robots can now do, plus things grippers never could. Hands are the new frontier; grippers are not. Hardware has come along faster than she expected. Reliability, repeatability, and cost still need work.
Strength varies by product. The Wuji hand is closer to a ten-year-old's, a bit weaker; the Shadow Hand can lift about 20 kilograms, and she has seen it open jars, though maybe not a very tight one. Some hands are built for lifting more; the Shadow Hand is also much bigger than a human hand, the Wuji closer to human-sized. On soft robotics she is interested in what is repeatable and durable, and in UMI-style gloves and tactile sensing. She jokes that mechanical-engineering roboticists sometimes tell computer-science roboticists they are not real roboticists. ICRA, she says, has every kind.
Labenz wonders whether alignment and safety, not research, will be the bottleneck on a domestic robot. Gopalakrishnan treats safety as a capability, not as something at odds with usefulness. People will not use an unsafe robot; the most useful agents will know what they need to do, not override human instructions, and not cause chaos. Robotics has layers digital AI often skips: operational safety — a humanoid that falls can injure someone out of clumsiness, not malice; guardrails while completing the asked task; and sensor failure. One demo puts a basket over the robot's head mid-task. Continuing blind is wrong; asking someone to remove the obstruction is right. Safety runs from mechanical design up through the ER brain, and as models get more capable she wants more FLOPs spent on it.
She also notices a double standard. A gripper robot that fails does not surprise anyone; a humanoid grappling around is judged much more harshly because it looks like a person. Researchers carry two brains: one knows these are machines, the other feels something human-like when the robot talks. Gemini Robotics 2's human-robot interaction is no longer preprogrammed nods. Gestures are generated in the conversation, and the launch videos include interviews with the robot. Asked the hardest part of packing in a garage, it said the videotape — "my favorite object" — was very hard to put in the basket instead of holding onto it. The film crew in the back laughed. Would a robot like a videotape? At least it will self-report.
The jury is still out on world models, and the data diet will stay a mixture
Labenz borrows a thesis from NVIDIA's Jim Fan: robotics models cannot just look at the present, reason, act, and loop. They need some forward simulation, a projection of what should happen, and a measured delta — closer to how brains work, and related to world models. Language models came surprisingly far on token-by-token prediction without explicit world models; maybe robots can too. Gopalakrishnan's answer is that the jury is still out. A roboticist, she says, is emotionally attached to the problem, not the method. VLAs, world-model policies, agent-style robotics — the recipes have not stabilized, which is what makes the field fun. Different schools can all turn out to be right.
Fan also bets that egocentric human video plus simulation will dominate, with teleoperation shrinking. Gopalakrishnan will not pick a single source. She plots data on scale versus precision. Teleop is highly precise — it is the robot's own controls — but not scalable and not future-proof: when the hardware evolves, the logs age even with good cross-embodiment. UMI-style handheld capture with sensors is cheaper to scale than a robot in the loop, and action labels come from sensors rather than noisy vision. First-person human video is the most scalable and good for broad semantics, but noisy, lacking fine hand signals, and confounded by humans who come in different sizes and habits. The likely path is a mixture, and the mix will change every time better hardware shows up.
On how a person should talk to many robots — each as its own entity, or one intelligence running a fleet — she wants a highly multimodal surface with both high-level and low-level access. Language for ordinary commands; fine-grained feedback when the robot is not cleaning the desk the way you like; low-level interfaces kept open for safety and interpretability. Gemini Robotics 2 already showed dual-robot collaboration: an Apptronik Apollo doing part of a task, stopping, and another taking over. Some jobs will be swarms you hand off; some will be one robot you talk to. Labenz hears the same pattern as modern software: an assistant that can do everything you can click, plus the ability to grab the mouse when the assistant is not getting it.
A freeze is safer than a collapse, and a family dog still has 25 trillion parameters
Labenz asks about "circuit breakers": an airbag in the torso you can puncture to kill the robot. Gopalakrishnan says the long-standing version is an e-stop. A hard e-stop cuts power and the body can collapse, which is itself dangerous; a soft e-stop freezes the robot, often safer for bipeds that are not statically stable. Newer manipulation work is about force at the end effector — stacking chips without crumbling them, handling delicate material, backdrivability and compliance. Sensing has improved enough that models can read those forces and make better decisions than they could without the signal.
She has not seen the fruit-fly-connectome robot demos on her Twitter feed. Zooming out, does she imagine a robot body of her own? Teleoperation already is that, in a sense: your physical body in one place, a robot body stacking shelves, sometimes across the Atlantic. A friend wants extra arms popping out of his back, like certain Indian temple statues, to pick things up as he walks. She also looked up her dog's neural-network parameter count: 25 trillion, more than a lot of the models out there, even though the dog's language skills are not that developed. The biological substrate robots are chasing is still that large. The direction she has been arguing all hour is nevertheless clear: one brain, any body, useful work first, racing later.
同频道的其他文章
1:25:56EN
2:17:48EN
2:00:51EN
1:26:46EN
1:10:12EN
1:28:08EN