The Robot That Folded the Laundry: Physical Intelligence’s Three Frontiers

Hausman says he had worked in robotics for about 15 years and for most of that time never saw a clear light at the end of the tunnel. The early approach was to write code that described what the robot saw and how it should move, then build a library of primitives task by task. The world proved too complex for that strategy, and roughly seven to ten years ago the field began shifting toward learning from experience. That shift was unsatisfying at first because it moved responsibility from human-written rules to algorithms and data, but it kept moving in a more end-to-end, learning-based direction.

Driess says older robotics research often trained a machine learning algorithm for a specific task, measured success, and moved on to the next task. Large language models suggested a different pattern: one model trained on many things at once could improve as data and tasks increased. The robotics question became whether one model could be trained across many tasks so that more data would yield broader capability. PaLM-E was one of the early projects aimed at that question.

Driess notes that language models alone did not work directly on robots because they could not perceive the world. Systems such as SayCan grounded a model through controllers that told it which actions could be executed. PaLM-E then integrated vision directly into the model and trained on robot data plus other data sources so the model could perceive and make decisions. The surprising lesson was that a strong pretrained backbone made these models more data-efficient than expected, because learning a hard task required much less data once pretraining was in place.

Hausman recalls the RT-2 experiment in which a single-arm robot moved a Coke can onto a picture of Taylor Swift; the demo was technically clumsy and the can tumbled, but it was a clear signal that the approach would work. The important part was not the execution quality but the connection between a pretrained vision-language model’s web-scale understanding and robot motion. The robot had never seen Taylor Swift, yet it could link that concept from internet data to an action that brought the can to the picture. This showed that robots would not need to experience everything firsthand if they could inherit knowledge from pretrained models.

Hausman says Google was an incredible research organization, but generalist robotics is so large and complicated that it needed an organization whose sole reason to exist was solving that problem. He and his co-founders concluded that a dedicated company was necessary if the resources and people could be assembled. Once it became clear that starting such a company was possible, the decision followed.

Physical Intelligence framed the problem around three axes: capability, generalization and performance. Capability meant getting robots to do tasks they had never done before—dexterous, long-horizon and complicated manipulation. Within the first five or six months, PI Zero demonstrated laundry folding, box building, bussing tables and other tasks across a range of robots. Hausman says the practical test became that if a task could be teleoperated, the model could probably be trained to do it.

Generalization was the second axis: could a model act correctly in a situation it had never seen? PI5 addressed this by bringing a robot into a new home, which the team treated as the hardest case because homes are so diverse. The robot could perform long-horizon tasks such as cleaning a bedroom, making a bed or putting dishes away. It did not work every time, but it worked some of the time, and the team began to understand what better generalization requires.

Performance is now the biggest challenge, and Hausman says it is less clear than capability or generalization. It is not obvious that more data or more diversity alone will solve it; he expects algorithmic work and new ideas will be needed. Today’s models are more demo-ready than deployment-ready because they still fail often. The performance threshold will depend on the task: some applications require speed and accuracy, while others can tolerate a robot folding laundry overnight as long as the pile is finished by morning.

Hausman says self-driving and robot manipulation are both hard and both would have huge impact, but they differ fundamentally. If all humans and other cars were removed, autonomous driving would become much easier; if all humans and other robots were removed, manipulation would be about as hard. A self-driving car makes real-time decisions with catastrophic risk, but it never has to touch or interact with the environment. Manipulation inherently involves contact, and contact makes the problem exponentially harder: knowing that a car should go from point A to point B makes execution simple, while knowing a shirt should be folded still leaves the hard part unsolved.

The shared difficulty is long-tail performance. Autonomous vehicles faced a vast diversity of rare events, and robot manipulation will face a similarly large long tail before widespread deployment. Hausman notes that autonomous driving was promised a decade ago, seemed to move slowly, and then arrived in places such as San Francisco. He expects robotics may follow a similar pattern: a long road that suddenly produces a “they’re here” moment.

Hausman pushes back on the common misconception that robotics is bottlenecked on hardware. He points to an old teleoperated robot demo that tidied a home and would still look state-of-the-art as a humanoid demo today. The caveat is that a person was controlling it with their own brain, which proves the hardware can already perform the mechanics. Anything that can be teleoperated shows that the bottleneck is intelligence, not the body.

Modern humanoid robots are impressive mechatronic achievements, and Hausman is glad the hardware efforts exist. But he does not think hardware is the current constraint; without the intelligence, the field remains bottlenecked on software. Hardware may become a constraint eventually, but it is not what is holding back deployment now.

Hausman calls the generalization milestone—bringing a robot into a new home—one of the most significant results so far. In the past, even impressive robotic demos required data from the exact same environment, and small changes such as lighting or a light switch could break the system. A new home introduces different tabletops, lighting, distractor objects and object placements, all of which require broader understanding. PI5 being able to operate in that setting was a major moment for the company.

Driess frames the result against a basic machine learning intuition: the model that achieves zero training error is a lookup table, but robotics cannot reset a scene exactly. For a long time, many people assumed that training on the test distribution must strictly improve performance. The PI5 paper includes a plot showing that training on a sufficiently large and diverse set of scenes can match the performance of a model trained on data from the unseen kitchen itself. Driess calls that a magical moment because the model generalizes without having seen that kitchen.

The next step is performance. Hausman says the team looks step by step at what blocks wide deployment: first capability, then generalization, and now robustness. There have been small-scale indications of how performance could be improved, but the unsolved challenge is a recipe that applies to any task at scale, regardless of hardware. Until that recipe exists, it is hard to predict exactly what the next performance milestone will look like.

Hausman says his own timelines were far too long: he expected three or four years to reach where the company is today. Driess believed in the large-model-plus-large-data scheme but emphasizes that many problems remain unsolved. He also notes that in machine learning, many people have ideas; the hard part is implementing them in a complex system with real data collection, hardware, model sizes and real-time constraints. Seeing those pieces come together faster than expected has been encouraging.

Driess says the initial hope after RT-2 was that robotics would ride the wave of improving vision-language models. That has not fully happened yet; better LLMs help through faster and cheaper inference, and VLMs help command robots in language, but the underlying LLM is not the main bottleneck. Visual capability remains a problem, and robotics still has cold, hard constraints around real-world data collection and hardware.

One open research question is whether generative video models will become useful for robotics. Driess says they have improved faster than he expected, but when conditioned on a robot scene they often produce unrealistic physics. If video models became accurate enough to act as simulators, they could speed generalization to new tasks and unseen environments by trying action trajectories before execution. That would not necessarily change the architecture, but it could provide a faster path to generalization.

Hausman wants a robotics-specific scaling law that maps dollars to model capability in a monotonic way. LLM scaling laws cover batch size, compute and data choices, but robotics needs an understanding of how spending on data collection, compute, video models or other inputs translates into capability. If that relationship were known, much of the problem would become an execution problem: allocate dollars and scale the recipe. The field is not there yet.

Driess describes robot data as time-series and multimodal: multiple camera views, robot states and actions, and language annotations that are themselves still a research problem. The data infrastructure is built in-house because it must be fast enough to store a growing dataset and flexible enough to support repeated iteration on annotations and formats. This differs from static datasets collected once, because robot data grows every day. Driess says data storage and data pipelines are important infrastructure problems, even if they sound boring.

Hausman says the team underestimated how much data robots generate. He also argues that robot interactions are unusually valuable because they are ground truth about how the world works: even a failed laundry fold teaches how the shirt deforms when the arm moves. Unlike LLM training data, where some examples are simply wrong and should be removed, robot failures still carry physical information. The volume and value of that data make infrastructure a central challenge.

Driess lists the machinery Physical Intelligence would like someone else to solve: deciding what data to collect, understanding data quality, doing quality assurance at scale, producing language annotations correctly, knowing how much more data a model needs, choosing collection strategies, training human operators, and giving researchers a bird’s-eye view of the dataset. These are not side tasks; they determine how fast the research loop can run. Managing data quality and dataset composition is part of the core problem.

Task selection is deliberately aggressive. Hausman says the team picks tasks that are hard enough to push capability—if a robot can fold laundry, it can probably assemble iPhones and do many other things—and tasks that are easy for the team to understand, deploy robots for and evaluate. Homes are convenient because everyone knows what a folded shirt or clean kitchen looks like, unlike working with a manufacturer’s process and schedule. Driess adds that laundry also answers critics who thought the Taylor Swift demo could be solved with an object detector and trajectory planner.

Driess says evaluations are important and very hard because a robot scene cannot be reset exactly the same way twice. The team must run enough evaluations and statistical analysis to know whether a difference is significant. It also learned that to show one policy is better than another, it often needs to run both across many tasks and many environments. Evaluation is one of the hardest parts of the research loop.

For subjective outcomes such as how crisp a folded shirt looks, Physical Intelligence gives operators guidelines. It avoids comparing absolute performance from one day to performance weeks later; instead it runs the baseline and the new model at the same time with the same operator and looks at relative ranking. Absolute numbers are used for tasks the team understands well, but relative comparisons help control for lighting changes, robot changes and operator pickiness.

The evaluation burden grows as models become more capable. PI must evaluate on more tasks and more robots, which means more evaluations to answer whether a model improved. Evaluations take a long time, are operationally heavy and have high variance. The team hopes simulation or video models can reduce that variance, but so far those tools have not been robust enough to give confidence that one model is better.

Driess says simulation has been extremely useful for locomotion because walking mostly requires modeling the robot’s own body. Manipulation is harder because the difficulty is modeling everything the arms interact with, not just the arms themselves. He sees evaluation as the first gate: if simulation is not accepted as an evaluation tool for manipulation, it has limited use beyond that. If it passes, diverse real-world data may help the model treat simulation as just another version of reality; Hausman says the team is not dogmatic and would switch if simulation became the way to go.

Hausman says running a research lab has no single recipe; the trade-off between exploration and exploitation depends on taste, belief and really good researchers. He looks for people with a track record of impactful results, but also for open-mindedness, willingness to change their minds and willingness to do whatever work the research goal requires, even if it is a data pipeline outside their formal role. Physical Intelligence exposes candidates to many interviewers and discusses hiring openly, and Driess says the team has so far avoided hiring mistakes.

Open sourcing is a deliberate choice. Hausman argues that robotics is still early, and the biggest risk is not a competitor getting there first but science failing to cooperate at all. To reduce that risk, Physical Intelligence wants to expose its ideas to as many people as possible and invite them into the approach it believes is right. Open models also help recruiting and let the team learn from external uses; PI Zero has been used on drones, surgical robots and autonomous cars.

Driess explains that fine-tuning a VLM only on robot data makes it forget capabilities and can train slowly. Knowledge insulation keeps training on web data, and tokenized actions let the VLM backbone adapt to robot actions without losing its pretrained abilities; flow-matching action experts are then bolted on with stopped gradients. This recipe sped training by 10x and helped generalization, which matters because data collection, training and real-robot evaluation form a slow loop. On latency, PI predicts action chunks and Kevin Black adapted image inpainting to fuse the chunk being executed with a newly predicted one while the world moves.

Looking ahead, Hausman hopes base models become more steerable and promptable, with efficient fine-tuning only needed to squeeze out the last percent of performance, speed or reliability; Driess expects post-training to remain but is unsure whether it will look like more demonstrations, guidance or self-exploration. In quickfire questions, Driess says he changed his mind about unseen-environment scaling—he once thought a million homes would be needed, but about a hundred sufficed—and about video models, which improved faster than expected. He calls generalist robotics models underhyped and humanoids somewhat overhyped. Hausman expects robots doing new tasks in deployment this year or next, and home robots within 5–10 years, or within 5 if progress continues; he imagines “vibe coding” for hardware and a home with no chores, freeing time for family.