Kay Ke on Physical Intelligence, Robot Learning, and the Future of Robotics

Zhang Xiaojun introduces Kay Ke, a researcher at Physical Intelligence (Pi), and notes that she also writes fiction. Ke says Pi is not focused on humanoid robots; she is interested in whether robots might eventually build or reproduce themselves. She summarizes Pi’s research progression as π₀ for capability, π₀.₅ for generalization, and π*₀.₆ for performance.

Ke says she would not have joined Pi if its goal were to build humanoids. She is more interested in robots that can eventually assemble themselves, which she sees as a possible milestone. She also summarizes Pi’s papers as π₀ for capability, π₀.₅ for generalization, and π*₀.₆ for performance; Xiaojun asks whether the company follows robotics in China.

Ke says she follows robotics in China closely because it cannot be ignored. Xiaojun then introduces her again and asks her to describe herself; Ke says she works on teaching robots to perform tasks and writes fiction in her spare time.

Ke says robotics research and fiction both require creativity and execution. Researchers must invent ways to handle tasks that current robots cannot yet perform, then implement those ideas; novelists likewise create stories that do not yet exist, one word at a time.

Xiaojun asks what Ke writes. She says science fiction is her favorite genre and that the present-day technology culture of Silicon Valley often informs her stories.

Ke is especially interested in two questions: how people will live after productivity changes, and how relationships will evolve in a very different future. Automating today’s work raises the question of what people will choose to do instead.

Ke sees human relationships as a timeless subject, even when a story is set in a technologically transformed society. She is introverted and often thinks best alone, but deliberately seeks conversations that challenge and refine her ideas.

During her PhD, Ke could often pursue an idea independently. At Pi, she has learned that building a general-purpose robotic brain requires a team whose members contribute different strengths; the work depends on closer collaboration than her earlier projects did.

Ke wonders whether coding agents such as Claude Code could reduce some day-to-day dependence on colleagues. A person can ask an agent about a module instead of immediately contacting its owner, and one person may now direct several agents as part of a team workflow.

Ke imagines personal agents filtering the information overload of the internet according to each user’s preferences. If people rely on bots to find and curate information, navigating the online world without one could become increasingly difficult.

Ke has imagined living alone in the countryside, as some YouTube creators do, but says she would not attempt it today. Modern life depends on infrastructure and chores that are difficult to manage alone; robots could eventually take on tasks such as home maintenance and plumbing.

If robots and agents continue to improve, a person may need fewer people to complete a project. Ke speculates that future productivity could make solitary life easier—even on another planet, where robots could build the infrastructure for an individual settler.

Ke says greater productivity could make people’s relationships looser, since individuals might survive with less help from others. She imagines that, in the future, a person could settle on a distant planet and rely on robots to build the necessary infrastructure.

Xiaojun imagines robots building infrastructure on a distant planet and asks whether Ke would want to live free of outside constraints. Ke says she wants her own quiet space, not necessarily a literal cave.

Ke says that after spending a long time alone, she would eventually want to come back out and see people. The pair then discuss whether someone might choose to fall in love with a robot.

Asked about stories of AI destroying the world, Ke says she cannot think of a convincing reason that a system built to help humanity would necessarily become its enemy.

Ke says she is trying to understand why that hostile-AI storyline should hold. She points to a recent example in which an agent given broad access deleted a person’s email, carrying out a task without matching the user’s real intention.

After a climbing injury, Ke searched for medical advice and considered asking an agent to review her information. She says she would still take its answer to a human expert because the agent cannot yet be held responsible for its advice.

Ke jokes that researchers can spend years teaching a robot to move a red block onto a blue one, a task a young child could learn quickly. She says human learning is much faster and that current agents remain far from the level needed to oppose people.

Ke says Pi’s unusual technical developments give her more concrete ideas for fiction, while her online writing can be unconstrained. Xiaojun jokes about protecting her pseudonym before asking about her childhood.

Ke says she grew up in Wuhu, Anhui, where educational resources were more limited than in major cities. Her city had an excellent informatics teacher, Jiang Tao, who returned home and taught programming after losing both hands in an accident.

Ke says she never met Jiang Tao face to face, but benefited from the systematic computer-education program he built. She credits his contribution with helping students like her discover informatics competitions.

Ke began informatics classes at about eight years old. At first, she was excited simply to use a computer and play with programming; later she explored other possible paths, including psychology and economics, before returning to computer science.

Ke says psychology interested her, but she wanted to solve concrete problems rather than spend years analyzing them. Economics helped her understand society, yet did not answer what she wanted to do in it, so she eventually returned to computing.

Competitions did not feel like intense pressure to Ke when she was young; they were enjoyable classes. She especially remembers LOGO, whose graphical interface let her make a computer draw pictures, and a C-language textbook she read closely.

Ke says programming first felt joyful: LOGO let her make a computer draw pictures, and she later read a C-language textbook closely. She thinks encountering programming as a child subtly changed how she approached problems.

Ke recalls learning programming ideas through stories—for example, recursion as a wish-granting genie. She says the lessons became part of her thinking, though they did not suddenly change her personality; Xiaojun asks whether she was a well-behaved or rebellious child.

Ke says her friends call her ‘Sister Ke.’ Xiaojun asks whether she has ever felt misunderstood or treated unfairly.

Ke says fairness is difficult to define and people understand it differently. She tries to preserve the fairness she can within her reach rather than insist that others do exactly what she wants.

Xiaojun asks whether a sense of injustice is what prompts people to push back against the world. Ke says fairness is hard to define and people understand it differently. She tries to preserve what fairness she can within her reach, rather than demand that others do exactly what she wants; she also thinks some values may be more universal. The subject can still leave her feeling heartbroken.

Ke says studying abroad made educational inequality more visible. In her home province of Anhui, only a few students per ten thousand might reach Tsinghua or Peking University, compared with far higher odds in some major cities. She was a diligent but uneven student who focused on subjects she liked.

Ke describes herself as a student who worked hard but was not exceptional across every subject. In a competitive class, she found it difficult to match classmates who were intensely focused on a single goal.

She hopes that new technology and productivity could eventually ease some inequalities in access to resources. Ke says she was not the most competitive student: she studied what interested her, rather than forcing herself to excel at every subject. Asked about rebellion, she recalls traveling in Southeast Asia.

In high school, Ke cut her hair very short before backpacking in Southeast Asia. The trip had no clear career purpose; she simply wanted to see the region, and her family mostly worried about her safety and urged her to return.

Ke says she went backpacking for about two weeks near the end of her second year of high school, drawn by Indian mythology and religious architecture. Her parents later came to check on her. The short haircut was also practical for travel.

Ke says she was not very concerned with grooming as a student, and a close female friend sometimes helped her get ready. She explains that looking more masculine felt like a way to protect herself while traveling alone as a young woman.

Asked where that sense of risk came from, Ke points to social messages and news that can make women feel more vulnerable. Xiaojun notes that she chose to adapt her appearance rather than abandon the trip.

Ke says disguising herself as a boy felt like a practical way to do something she wanted despite feeling unsafe. The conversation then turns to how she unexpectedly decided to study abroad.

Ke says she did not follow the conventional route through university. She wanted to explore what she could do and find work that felt meaningful.

She studied mathematics and physics and resisted being defined by what she was not good at. Xiaojun asks about her early interest in games, leading Ke to describe Rainblood.

Ke says the game Rainblood was created by a Tsinghua graduate who later studied at Yale. Its story and world drew her in, and Xiaojun asks how she became interested in games and art.

Ke says she was drawn to both mathematics and physics, while also exploring other interests. She recalls being curious about the games she played and their creators.

She decided to study abroad and continued playing games. Xiaojun asks why she was drawn to Rainblood and its artistic world.

Ke describes being moved by Rainblood’s art and story. She says games can make imagined worlds feel vivid, and recalls being especially affected by the experience.

Ke says games let people inhabit unfamiliar roles and experience different lives quickly. She remembers lines from Rainblood that made the game’s revenge-driven world feel vivid, even though she no longer recalls the exact wording.

Ke says her college majors helped her write fiction, but she eventually found greater fulfillment in research: doing something difficult that others could not, or doing it better.

Asked whether she experienced culture shock, Ke says the change was significant. In China, familiar social relationships had felt natural; moving abroad made her reconsider how she fit into society.

Ke says moving into a new environment helped her examine her earlier social surroundings more objectively. She also found that American universities encourage students to express ambitions and rally others around them, which pushed her to speak up despite being introverted.

Ke says she became much more comfortable talking with people than she had been in high school. Xiaojun asks how China and the United States differ in their social and cultural foundations.

Ke says injustice exists in both countries, but China had experienced rapid change during her parents’ generation. Her later time studying in Britain gave her another point of comparison.

Ke describes Britain as having especially entrenched class distinctions, with social class shaping relationships and taste. She sees the United States as less rigid, while Silicon Valley tries to leave older status markers behind and build new things.

Ke says she had no grand purpose when she left school. Over time, she discovered that she wanted to work in technology, an industry she still cares about.

She studied economics in college and found the subject interesting, but felt that much of the work involved combining numbers and producing reports. She began exploring more concrete ways to use her skills.

When Xiaojun asks what a first-year student could put on a résumé, Ke describes helping a high-school classmate develop small software after programming competitions. Even a small project can demonstrate useful skills and reliability.

As a college freshman, Ke joined a computer group and met Li Bo, who became her first academic mentor. Ke guesses that her economics background and interest in game theory helped connect her to Li Bo’s work on adversarial machine learning.

Ke says a first-author paper from that period later helped with her doctoral applications. Xiaojun asks what attracted her to game theory.

Ke explains that game theory studies how agents with competing goals interact and may reach an equilibrium. She connects this idea to market competition and to machine-learning systems that learn through adversarial interaction.

To explain generative adversarial networks, Ke compares the generator to someone making counterfeit money and the discriminator to a judge deciding whether it is real.

As the generator and discriminator compete, the generated images can become increasingly realistic. Ke says this game-theoretic pattern has broad applications in machine learning.

Ke says her research direction changed during her PhD. Her first paper drew on game theory, but she found that she preferred applications and moved toward robotics.

Her robotics work began with imitation learning and later shifted to reinforcement learning. She says copying examples alone can limit innovation, while reinforcement learning lets a system explore and push beyond its previous performance.

Ke entered a lab whose work was rooted in traditional robotics. Around 2017, that field emphasized path planning and control: plan a sequence of actions, then control the robot so it can carry them out.

Ke says data-driven robotics was a niche interest in her lab, and she was the only student pursuing it. Her advisor specialized in traditional planning and control, but trained students to understand the full robotics stack.

Her advisor was open to machine learning while encouraging her to study how earlier robotics methods solved each task. Ke says their discussions helped her connect machine-learning ideas with prior approaches.

Ke says she sees a broader mission in machine learning for robotics: reduce how much behavior must be specified by experts and let machines learn from data. Xiaojun asks about her advisor, Sidd Srinivasa.

Xiaojun asks why Ke joined Sidd Srinivasa’s robotics lab. Ke says she was drawn to methods that can optimize a well-defined task without requiring researchers to model every mechanism inside the system.

Ke says robotics was still a niche and expensive field when she entered graduate school, while NLP groups were already investing heavily in datasets. She describes the high costs of robot hands and the difficulty of repairing them.

In 2017, a dexterous Shadow hand could cost hundreds of thousands of dollars, and even a Barrett hand cost more than $100,000. Repairs were difficult, so researchers sometimes had to fix the hardware themselves.

Ke says Franka became one of the more affordable robot arms, though it still cost tens of thousands of dollars. She entered her PhD program in 2017, and Xiaojun asks her to review the field’s history.

Ke says her account is selective, then starts with Carnegie Mellon’s robotics institute, founded around 1979. Its work combined perception, decision-making, and action to move robots into real environments.

CMU’s work branched into applications such as disaster response and autonomous driving. Ke says the DARPA challenges of 2004 and 2005 energized the field and helped stimulate later autonomous-driving companies.

Ke turns to CMU’s later work in manipulation and control. She identifies Matt Mason, whose research connects robot movement with how a controller guides even a simple mechanism.

Mason argued that dexterity comes less from elaborate fingers than from the brain controlling the body. Ke says this idea influenced her own PhD work on using chopsticks as a simple but capable manipulator.

Ke traces another branch of legged-robot research to Sangbae Kim, who built the MIT Cheetah. She also mentions Marc Raibert, a contemporary of Mason who founded Boston Dynamics and advanced legged robots.

Ke sketches the evolution of machine learning, from early work associated with Marvin Minsky to the Andrew Ng courses that introduced many students, including her, to the field. She says her first AI lab also reflected different research traditions.

Ke describes Peter Abbeel’s influence on reinforcement learning and robotics. His student Sergey Levine later co-founded Pi, where Ke works on reinforcement learning.

Ke learned much of reinforcement learning from Levine’s lectures, discussions, and papers while she was a PhD student. She values researchers who can move from one field into another and bring ideas with them.

Ke says people at Pi often discuss research together. She describes Levine as a science-fiction fan and a gifted storyteller whose papers often include literary references and memorable titles; colleagues jokingly called his editing ‘SergeyGPT.’

Ke describes Chelsea Finn as disciplined, intuitive, and perceptive about robot actions. Finn’s judgment shaped Pi’s data-collection choices, and Ke says she pushed much of the work that led to π₀.

Ke says Finn and Levine complement each other. She then mentions Fei-Fei Li’s students, NVIDIA’s robotics research, and Shuran Song, whose work helped develop diffusion policy.

Ke names ACT and ALOHA as influential work by Chelsea Finn, then points to Deepak Pathak and Abhinav Gupta as visible figures in machine-learning robotics. Gupta’s background in visual arts informs his unconventional research ideas.

Ke recalls Gupta’s idea of using a cheap pole-mounted gripper to collect data outside the lab, an early cousin of today’s UMI approach. Pathak’s curiosity-driven reinforcement-learning work later led him and Gupta to co-found Skild AI.

Ke says robotics researchers tend to believe in their chosen approach, but the field’s favored methods change over time. She remembers that traditional robotics was still dominant when she began attending conferences.

Ke says she believed in her own research direction despite working in a lab whose main focus was different. Her advisor challenged her to show that machine-learning methods worked on real robots, not only in simulation.

Ke says her advisor pushed her to consider whether emerging machine-learning work would translate into practical performance. That influence shaped how she evaluates research, even though she and her advisor preferred different approaches.

Ke jokes that machine learning appealed to her partly because it can produce results without requiring every mechanism to be modeled. Her advisor preferred elegant, well-understood solutions; Ke wanted to investigate whether a learned method could be made reliable enough.

Ke defines black-box optimization as searching for a solution to a well-specified task without requiring a full account of every internal mechanism. She argues this can make useful capabilities accessible to people who lack specialist knowledge.

She says practical results matter, while traditional methods still provide useful insights. The two approaches can be compared by how much effort they require and what performance they deliver, rather than treating one as universally sufficient.

Ke notes that Boston Dynamics achieved striking feats such as backflips, but required extensive engineering to tune each task. Learned methods may reduce the engineering effort and could eventually make it easier for ordinary users to interact with robots.

Ke says robots today can often perform a task only after experts inspect failures and adjust the system. The longer-term aim is for ordinary people to interact naturally with a robot and get the result they want.

Ke says legged robots can be easier to obtain than robotic arms, but they raise different control challenges. Modeling friction for every surface is difficult; a dog-like robot can rely more on repeated contact and feedback, while precise manipulation demands more detailed physical knowledge.

She explains that a robot’s interaction with the ground can be learned through contact rather than fully modeled in advance. This distinction helps explain why traditional control methods have succeeded on some tasks while leaving other settings challenging.

Asked when machine learning became more mainstream in robotics, Ke says the shift took place around the pandemic. Her advisor had challenged her to move beyond simulation, so she built a chopstick robot from components and wrote its system software and drivers.

Ke says her work helped colleagues see how data-driven methods might apply to other projects. For example, a wheelchair-mounted arm could feed someone, and learning could help it choose how to grasp and deliver food rather than relying entirely on hand-coded paths.

Her lab gradually began using neural networks not only for perception but also to let robots learn their own actions. Ke chose chopsticks because they make manipulation difficult: if a system can succeed with two contact points, other tools may be easier.

Ke recalls her advisor’s provocative claim that chopsticks could solve most tabletop manipulation tasks. The challenge pushed her to build a robot that could perform a genuinely difficult action.

Ke says her advisor liked simple manipulators and challenging tasks. She proposed building a robot around a difficult manipulation problem, rather than continuing with a conventional setup.

Xiaojun asks how the project developed. Ke says they chose a specific task and began with imitation learning, while asking whether a robot could learn useful abilities from data.

The hardware was imperfect, but Ke says a data-driven method could still work. The team wanted to see whether learning could handle the real robot rather than relying on an idealized setup.

She describes a ball-picking task with uneven joints and mechanical backlash. The robot had to adapt to these imperfections instead of assuming that every movement would be exact.

Ke says the project eventually used reinforcement learning, letting the robot improve through repeated attempts and exploration.

She describes the team’s work as a practical test of whether learning methods could make a robot more capable on a concrete task.

Xiaojun asks about tensions between traditional robotics and machine learning. Ke says people often trust the methods they know best, which can make new approaches seem less credible.

Ke says it is useful to validate a method on a well-defined task first, then ask whether it can generalize. A clear initial problem makes it easier to assess what has actually improved.

She says robotics is too important to approach as a contest between camps. Progress depends on making systems robust enough to work beyond a carefully controlled demonstration.

Ke favors interpreting other researchers’ work charitably and collaborating across fields. She says different groups bring useful expertise to robotics.

Asked whether robotics research is geographically concentrated, Ke says machine-learning groups have clustered around Stanford, Berkeley, and Silicon Valley, while traditional robotics has deep roots on the East Coast. Researchers from both regions now join Silicon Valley companies.

Ke says researchers are experimenting with new tools, while traditional robotics experts can offer practical knowledge about hardware and specific projects.

She notes that machine-learning researchers sometimes study hardware such as motors, guided by traditional professors. Xiaojun asks about the academic backgrounds of robotics startup founders.

Ke describes Pi and Skild AI as startups with academic roots and ambitions to build general-purpose robot brains. She says Pi has focused more on two-arm manipulation, while Skild has explored locomotion and humanoid or quadruped platforms.

Ke says humanoid control remains an open problem and companies emphasize different capabilities. Tesla helped draw attention to humanoids, while Figure and 1X have their own founder backgrounds and technical approaches.

Ke says Dyna’s founders combine entrepreneurial and technical experience and appear focused on commercial deployment. She suggests their work may help show how learning-based robots can create value in homes and other real settings.

Xiaojun calls 2023–25 a boom period for robotics startups. Ke says ChatGPT changed priorities at large technology companies, prompting cuts to robotics teams and some researchers to leave Google and Meta to start companies.

Ke names Generalist AI and Sunday Robotics as companies that emphasize UMI, a handheld device for collecting demonstrations in everyday settings. Their work suggests that data collected with the device can be transferred to a robot.

Ke thinks Generalist may lean more toward industrial applications, while Sunday’s public videos show home settings and a friendlier-looking robot. She cautions that company strategies change, and says Tesla has strong hardware resources while Pi focuses on the robot brain.

Xiaojun asks whether Pi and Skild are brain companies, like Google and Tesla. Ke says Pi focuses on the robot brain and Skild’s work is less visible; Tesla has substantial hardware resources, but the startup landscape can change quickly.

Ke says robot hardware is difficult and expensive to develop, whereas software is easier to enter. Tesla may benefit from its experience building cars; Figure uses motors, while 1X has explored cable-driven actuation.

Ke says Generalist, Sunday, and Dyna are also likely to make hardware decisions. A machine-learning team may need to build its own robot because off-the-shelf hardware rarely fits its application perfectly.

Ke says hardware still has a long way to go. Her practical ideal is a reliable robot that could be sent into someone’s home and consistently handle everyday tasks.

Ke recounts how Boston Dynamics founder Marc Raibert came to wear Hawaiian shirts. After someone criticized his casual clothes at a meeting, he embraced the look and made the shirts part of the team’s public image.

Ke describes Abhinav Gupta as imaginative and unpredictable, and mentions his former student Xiaolong Wang, now a professor at UC San Diego. She says conversations with Gupta can take unexpected turns.

Asked which researchers she admires, Ke says she appreciates many people’s strengths but is especially drawn to game creators and novelists. She separates work, which she sees as optimization, from the human stories she values in life.

Xiaojun asks how robotics researchers differ from people working on large models. Ke says robotics requires collaboration across disciplines and attention to how the physical system produces its data.

Ke contrasts asking for a million more examples with inspecting the data and the robot’s hardware conditions. Roboticists often want to see what the robot actually did.

She adds that researchers with large-model backgrounds may also help break out of familiar assumptions and ask what information is needed to make progress.

Xiaojun asks how NVIDIA, Google, and Tesla differ in their robotics bets. Ke says she can speak only from public information and her own perspective.

Ke says Tesla appears especially committed to humanoid robots. She believes hands may eventually be useful, though they are not essential for every task today.

Tesla seems committed to a robot whose overall form resembles a human. Ke says Google may connect robotics to Gemini’s multimodal abilities, including spatial perception and planning.

Ke jokes that NVIDIA does everything to sell more cards, but says abundant compute can help some projects. She adds that researchers should also make the most of their own teams’ strengths.

Ke says large amounts of data and compute can improve results, but choosing the right problem and using a team’s strengths also matter. Xiaojun jokes about whether NVIDIA researchers get more funding.

Xiaojun asks how companies differ in their hardware and software choices. Ke says hardware details are not usually discussed openly and often have to be inferred from public videos.

Ke says companies keep much of their hardware and software confidential. Pi is relatively open because publishing research remains part of its academic culture; she describes its office as less restrictive about visitors.

Ke says a visible difference among companies is whether they prioritize humanoid or non-humanoid bodies. Tesla, Figure, and 1X have made strong bets on humanoid form.

Supporters of humanoids argue that human environments are already designed for a human-shaped body, which could make such robots versatile. Ke says she is both practical and imaginative.

When Xiaojun asks whether she is practical or imaginative, Ke says she is both. She does not think robots have to be humanoid to perform many useful tasks.

Ke says early robots should first become capable and reliable at manipulation. A robot need not have a human shape to perform useful work in the real world.

She compares humanoids with cars: wheels are not found in nature, yet people redesigned roads and cities around them. A useful robot form could eventually reshape its environment too.

The conversation considers whether robots might benefit from forms evolution never produced, such as extra arms or multiple wheels. Because robot bodies can be replaced, people can imagine configurations beyond the human template.

Ke says she is imaginative about robot design but wants current configurations to work first. Xiaojun asks why Pi’s goal of a universal brain does not imply choosing a humanoid body.

Ke says robotics has a long way to go and identifies evaluation as one unresolved challenge. Unlike language benchmarks, physical tasks have to be run on a real robot, where hardware differences can affect results.

She compares robot evaluation with early natural-language benchmarks, where people had to judge generated text. For a robot, researchers must watch it perform the task, while accounting for physical limits and small changes in hardware.

A cup-picking test should vary the cup’s location and angle, lighting, background, and table height; it may also test objects on a raised surface. Ke says these many initial conditions can affect performance.

Ke says robotics lacks a simple leaderboard that shows which system is best. Companies have internal evaluations, but their tests may focus on different things, so the field needs more robust and comparable assessments.

Ke thinks the field is still exploring how to make evaluations broad and robust. Robotics has no settled definition of the frontier yet, and the work may look increasingly chaotic before researchers find clearer lines of progress.

Asked how a robot is built, Ke says her account begins once a motor has been made. A motor becomes a moving joint, and several joints connected by structural parts form an arm.

Ke explains that builders assemble joints into an arm, then add electronics, drivers, and a control system. At the lowest level, commands such as rotating a joint to a given angle make the hardware move.

A controller translates a desired joint position into a signal the hardware can execute. This low-level control is one part of the system; deciding which movement will accomplish a task belongs to a higher level.

Ke describes a hierarchy: a higher-level system decides what the robot should do next, while lower-level controls translate that decision into physical movement. The robot then observes the changed scene and plans another step.

Ke says Pi’s main research story is clearest through its three major papers: π₀ focused on capability, π₀.₅ on generalization, and π*₀.₆ on performance. The team began with the question of whether a large-model architecture could control robots at all.

Ke says the team began π₀ in early 2024, excited by the possibility of using a large-model architecture to control a robot but unsure what it could actually do. Pi wanted to make something that had not previously been possible. She says π₀ was designed around three tasks, beginning with one already familiar to the audience.

Ke describes three early π₀ tasks: folding clothes, stacking boxes, and bussing items from a table. The results surprised the team because they showed that a large model could handle manipulation tasks that had previously seemed out of reach.

A model may work well on the conditions represented in its training data but fail elsewhere. π₀.₅ focused on this generalization problem, testing whether a robot trained in one home could adapt to many other homes.

Between π₀ and π₀.₅, the team published work that improved the model’s language and hierarchical capabilities. They collected data in many homes, including Airbnbs, to evaluate performance beyond the controlled setting of an office or factory.

Ke says the team tested the model across roughly a hundred homes and observed improvement as the number of environments increased. The results suggested that broader data could help, although gains may eventually diminish.

Ke says π*₀.₆ addressed another core question: a model that can attempt many tasks is not useful if it performs each one poorly. The team therefore shifted its focus toward improving performance.

As part of the reinforcement-learning team, Ke worked on π*₀.₆ to improve task success. One idea was to let an agent perform a task, collect experience from its own attempts, and learn from those outcomes.

Ke says experience data can help a robot improve beyond a fixed set of demonstrations. Instead of relying only on a human operating an arm to show each movement, the robot can practice and collect data from its own actions.

Ke believes robots will eventually generate much of their own data through interaction with the environment. She expects robots to become cheaper and more accessible, reducing some current concerns about the cost of data.

Collecting real-robot data is expensive: a team needs a robot platform, must install and maintain it in a home, and often has to hire people to carry out tasks and record the results.

Ke says using a large model could lower the cost of collecting data. She adds that demonstrations containing mistakes and the corrections that follow can also be useful training experience.

Small errors can compound over a long sequence of actions and push a robot into a situation it never encountered during data collection. A capable system therefore needs ways to recover when its plan goes wrong.

Ke uses the saying ‘a cat that catches mice is a good cat’ to emphasize practical results: researchers should trust whichever data proves useful. She still favors real-robot data as the most direct evidence of how a machine behaves.

Ke says simulation alone may not solve every task because of the sim-to-real gap. Even modeling clothes is difficult: their softness, friction, and folds are hard to reproduce faithfully.

Some researchers believe these simulation problems can be solved, and are actively working on them. Ke says that, as of the recording, deformable materials such as clothes remain a difficult frontier.

For folding clothes, Ke says the most straightforward approach is still to collect experience on a real robot. Open models can provide a starting point, but teams need to test and improve them on their own tasks.

If a project requires a large and varied dataset, Ke argues, teams should seriously consider investing in real-robot data rather than relying only on simulation.

Ke says data must be relevant to the task and its value needs to be studied. Real-robot data is difficult to replace, though some tasks can work well without it; researchers should choose the data source that fits the problem.

Ke helped design strategies for Pi’s generalization tests. For folding clothes, the question was whether a model could handle different garments and conditions rather than simply repeat a familiar demonstration.

A robot that has folded one known garment may encounter something outside its training data. Ke says researchers need to test whether it can perceive the new item, begin the task, and continue successfully.

Ke says task evaluation must define both what counts as success and how long the task takes. Even a seemingly simple task has many hidden conditions that affect whether a robot should be judged successful.

For π*₀.₆, the team compared its researchers’ performance with that of data collectors who were more accustomed to operating the robot. This raised questions about how to design fair, repeatable benchmarks.

Xiaojun asks about other influential robotics papers. Ke says research advances through successive waves: each group builds on earlier work and contributes another step, even when the result is not a complete solution.

Ke describes HiRobot, a project on human-robot interaction. If a robot receives only an initial instruction and must act for ten minutes, it may lose track of what to do; the work explored using intermediate feedback to keep the task on course.

Ke recalls Pi’s internal ‘Olympics,’ where researchers and data collectors formed teams for informal robot challenges. The events let them try tasks and see which actions a model could learn from demonstrations.

She says the challenge was to use human demonstrations and a large model to accomplish actions that had previously seemed difficult for robots. Xiaojun asks what the hardware team contributes to this work.

Xiaojun asks what the hardware team can contribute. Ke recalls struggling for more than twenty minutes to open a door with a robot; after the team optimized its strategy, the robot could complete the task autonomously.

Ke describes Pi’s partnerships with companies working on clothing-folding and packaging robots. Their robots run models trained by Pi, allowing the company to advance research while helping partners pursue commercial applications.

Ke contrasts today’s coding agents with earlier research, when people often needed specific contacts or technical skills just to get started. Tools such as ChatGPT and Claude can now produce code quickly, and she hopes assembling robot hardware could become similarly accessible.

Xiaojun asks how deeply Pi builds its own hardware. Ke says some details cannot be disclosed, but the hardware used for π*₀.₆ was made in-house and optimized for stability and the task.

Ke gives the gripper as an example: a simple hand may not suit every job, so a robot could use interchangeable grippers for different tasks, such as making coffee.

Xiaojun asks whether companies should optimize hardware for software. Ke says hardware should be optimized for the mission and the performance required, while noting that no off-the-shelf platform is ready for every use.

Xiaojun asks whether Pi wants to become the OpenAI of robotics and how π*₀.₆ compares with GPT. Ke says the goal is to build a universal model, but the team still sees shortcomings and works to improve the system.

Ke says the starting points for robotics and language research differ. She recalls dismissing early language generation as merely predicting likely words, then seeing researchers make progress with reinforcement learning and better benchmarks.

Ke says natural-language evaluation was initially vague, though datasets and benchmarks gradually made it clearer. She was drawn to robotics because robots can attempt many different tasks, which makes it possible to define new challenges.

Ke says language is a natural way to tell an agent what to do, and she sees potential for the same kind of interaction with robots. VLA systems are still being explored, but language could help specify tasks.

Xiaojun asks whether competition among large-model companies stimulates research. Ke agrees, while noting that companies may eventually differentiate themselves through the products and applications built around their models.

Ke imagines a division of labor in robotics: some groups may build general robot brains, while others focus on commercializing particular environments and tasks. A product could even combine a brain from one company with another company’s application.

Xiaojun asks whether a robot brain could be a standalone system, trained specifically for robotics, or whether it will be part of a multimodal model.

Ke says a dedicated robot brain is possible, though some researchers may combine many modalities into one model. Such a system could eventually handle language, images, video, and robot control.

Asked about scaling laws, Ke says the question has many possible measures. She cites π₀.₅’s tests across homes: collecting data in more homes helped somewhat, but the gains were not dramatic, and the team is still studying how much data improves generalization.

Ke describes Pi’s work culture as spontaneous: colleagues often begin discussing research after dinner, and many come to the office on weekends. The conversations can continue as new ideas arise.

Asked about a typical workday, Ke says it varies. Some employees commute for hours from the South Bay; she is a night owl and tends to wake later.

Ke says she often gets home around one or two in the morning. Pi does not require clocking in, and researchers with family responsibilities can sometimes work from home.

Ke says Pi has grown to around seventy people and may need to adjust its management as it expands. The company has no rigid daily meeting schedule; teams meet around particular topics.

Xiaojun asks about finding an office in San Francisco. Ke says it is difficult to find a space that suits a robotics company’s needs for hardware, robots, power, and network access.

Ke says Pi has an open research culture. She and a colleague organize reading groups and exchanges where people can discuss projects, including π*₀.₆.

The host jokes that whoever uses Claude Agent best should come onstage so everyone can copy their workflow. Ke says she was working on something else when colleagues first showed her the tool.

Ke says AI tools can make a person’s output feel three or four times larger, but research still requires running experiments, seeing where they fail, and adjusting. Progress takes more than firing off attempts without analysis.

Ke says CEO Karol Hausman manages much of the company’s infrastructure and helps research move efficiently. He also handles questions around how the work may be commercialized.

Ke says humanoids are worth studying, but she would rather focus on making robots perform tasks well. She worries that centering research on humanoid form could distract from the problem itself.

Ke says she is especially excited by pet robots and would like to build one someday. She sees companionship and emotional value as meaningful reasons to create robots.

Ke describes herself as valuing both practical results and careful evidence. Years of working with her advisor taught her to read papers skeptically and look for claims supported by experiments.

She says she and her advisor’s former student Abhishek had compatible interests. Sergey Levine encouraged Abhishek to speak with her as he was graduating, which helped bring the two researchers together.

Ke recalls visiting Pi while traveling around the Bay Area, soon after the company was founded. She thought she was stopping by to learn about the company, then realized the team was treating the visit as an interview.

Ke says she was surprised to find the Pi researchers asking her questions as though she were the interviewer. The informal meeting nevertheless gave her a sense of what the team was like.

Ke wanted a role that matched her research interests, and found Pi’s team serious about seeking truth. The company made her an offer while she was still considering academic positions.

Ke says she had imagined first joining academia, building a research group, and leading students. Before accepting a faculty offer, she began asking herself what kind of research she most wanted to pursue.

Ke ultimately declined the faculty offer and joined Pi. She says the decision came after reconsidering the kind of research environment she wanted, not because the academic path lacked appeal.

She says she was attracted to the possibility of doing research with broader scope than an individual academic lab could manage. Pi’s startup setting offered room to pursue ambitious projects with a team.

Ke says Pi’s open communication lets people raise an idea one day and potentially turn it into a project the next. She gives π*₀.₆ as an example: the team changed direction after realizing it needed more evaluation data.

Xiaojun asks why Ke turned down a Cambridge position at the last minute. She says the decision required weighing the different rhythms of academia and industry.

Ke says academic projects often unfold within a student’s several-year period, while industry can offer greater access to compute, data, and hardware. Those resources mattered for the kind of robotics work she wanted to pursue.

Ke says Pi’s collaborative setting was its strongest attraction. She did not want to work at a large company; she wanted to join a team and build something together.

Ke says she was not primarily anxious about graduating. She was trying to understand what she wanted to learn from one project and what question she would want to tackle next.

Ke had interned as an undergraduate at Google, where she found the experience enjoyable. But Pi’s startup environment offered a different pace and a chance to discuss ideas directly with colleagues.

At Pi, Ke can raise an idea with a colleague and see it become a project quickly. She gives π*₀.₆ as an example: the team’s plan changed substantially when it realized it needed more evaluation data.

Ke says reinforcement learning is her current focus, though she may change directions again. She sees research as exploration rather than a permanent commitment to one topic.

Ke compares reinforcement learning with personal improvement: begin with a specific action, practice, and learn from the outcome. She says the process of finding a better strategy can matter as much as the initial task.

Ke says she is exploring a broader direction that may also help make systems more general. She notes that researchers are already using reinforcement learning to improve large language models.

An exploration module could help an agent try actions it has not performed before. Ke says this remains an important, open problem in robotics and is not yet a major part of Pi’s work.

Ke says π*₀.₆ raises the challenge of identifying which part of a long robot deployment led to a good result. The system needs to connect a successful outcome with the actions that caused it.

An agent may make many attempts before one succeeds. Ke describes attribution as finding the useful experience within a large dataset so the robot can repeat what worked and learn from it.

Ke says reinforcement learning depends on defining the problem, a foundational issue that textbooks often gloss over. In language models, the prompt usually describes the task; in robotics, the same instruction may allow many valid outcomes.

Xiaojun asks what it means to tell a robot to fold clothes, since the instruction does not specify how the garment should be folded.

Ke explains that a robot can exploit a flawed reward, much like a game-playing agent that discovers a bug and uses it to win. A high score does not necessarily mean the robot did what people intended.

Ke says reward functions can express preferences through higher or lower scores. In language-model research, some tasks can instead be verified by a program—for example, whether generated code runs successfully—without relying on a subjective reward.

Robotics often lacks such clear success criteria. A cup can be moved to another table, but many physical tasks have more subtle outcomes than simply reaching a defined end position.

The host asks about folding clothes. Ke notes that people use many different strategies, and it is difficult to describe in words which method should count as the right one.

Ke says her daily use of Claude Code changed her view of natural-language interaction. She used to think that, as an introvert, she would prefer communicating without speaking, but she now sees the value of conversational tools.

Ke says an agent may not understand an instruction immediately, but a person can add context, ask it to search, or have it make a plan. Conversation can therefore be refined over several turns.

Ke says current vision-language-action models remain primitive. Language can state the task, but it often cannot capture the fine details of movement or the many ways a person folds clothes.

Asked whether robotics researchers agree on the best data, Ke says she cannot speak for everyone. Some believe simulation can be made substantially more useful, while others take different approaches.

Ke describes UMI as a way for people to collect robot demonstrations naturally in their own homes. It can be faster and easier than installing a robot in each home and collecting data through the robot itself.

Ke says there is no single agreed definition of good robot data. Its value depends on the task and on the quality of the actions it records.

A data sample may show an exceptionally precise, athlete-like movement, or a slower attempt with extra steps before reaching the object. Ke says movement quality is one dimension of data quality.

Ke says labels can reveal information hidden in an observation, such as where a glass is and what it looks like. Large-model teams may emphasize broad zero-shot capability, while application teams often focus on specific settings.

Ke says broad capabilities and task-specific methods can complement one another. Their goals may look different, but they can contribute to the same system.

A model that performs well in one setting may also improve in similar scenarios. Ke says better data and a stronger model can help with both the original task and related tasks.

Ke says that if a large model improves steadily on new tasks and environments, researchers can apply more targeted methods to refine it. She sees general improvement and task-specific work as mutually supportive.

Ke says Pi is trying to maintain a balanced research program rather than focus on only one of these approaches.

Asked about future robot architectures, Ke says some researchers expect world models to be important, but there is no settled definition of what a world model must be. The field needs continued exploration.

Ke sees Pi’s work as a progression from capability to generalization and then performance. Each phase has produced useful findings, though she expects many important robotics results to come from other groups as well.

Ke contrasts what robots could do in early 2024 with their abilities by late 2025. Folding a load of laundry once seemed unrealistic; later, it became a task that a robot could sometimes complete.

She says progress has made some previously uncertain behaviors visibly achievable, even if success is not guaranteed every time. Pi’s π₀.₅ and π*₀.₆ work explored those capabilities from a practical perspective.

Ke says the team tests whether a robot can apply what it learned in one home when it is placed in another. In-house results matter only if they can survive deployment in unfamiliar settings.

Xiaojun mentions Google’s Gemini release as an example of a system that may inspire robotics. The conversation turns to what demonstrations and capabilities might emerge in 2026.

Asked whether robotics is more like autonomous driving or large language models, Ke says it resembles large-model research more closely. Autonomous driving has its own complexity, but robot manipulation involves a broad range of physical tasks.

Ke says autonomous-driving systems face difficult high-level decisions, while robotics also has a complex low-level control problem. Manipulation requires coordinating physical actions in settings that are hard to model.

Ke says people tolerate small errors in language models because they know the systems can be wrong. Robots still need careful evaluation, though she does not think their standards must simply copy autonomous driving’s.

Ke says robotics and autonomous driving have developed in parallel, though both can learn from each other. Robotics can draw on autonomous driving’s experience with planning, deployment, and safety.

Ke says robotics and autonomous driving are parallel fields with different technical challenges, but can learn from one another. Robotics researchers can draw on autonomous driving’s experience with deployment, safety, and operating fleets of vehicles.

Ke says leading companies need their own simulators, while specialized vendors may provide tools others can rent. Autonomous driving has amassed far more data than robotics, and shipping a robot is becoming a key goal for companies.

Asked whether robots can do household work today, Ke says many home tasks remain unsolved. She identifies the complexity of homes as a major research challenge.

Ke says every home differs, and the combination of varied environments and chores creates a demanding generalization problem. Even a task such as folding a quilt may fail in a new home.

Ke says a home robot must be reliable and safe around people, not merely work for a short demonstration. She worries about a humanoid falling in a home, damaging the floor, or injuring someone, and wonders whether a smaller, lighter robot might feel safer.

Asked about China’s robotics progress after the Spring Festival Gala, Ke says the demonstrations made her excited. She sees China’s manufacturing supply chain and hardware strength as major advantages that can support faster iteration and commercialization.

Ke says Pi is unusually focused on research rather than immediate commercialization. She connects this choice to Peter Abbeel’s earlier robotics startup, Covariant, whose shift toward logistics may have diverted effort from building a more general robot model.

Ke says Pi has learned from that history and wants to solve foundational research problems before closing a commercial loop. She notes that some Chinese companies may be more pragmatic about earning revenue, while asking whether Pi’s limited hardware investment could become a disadvantage.

Ke says Pi cannot invest in every area without limit. She jokes that the team envies the funding available to companies such as OpenAI, since more investment could improve software, algorithms, compute, and data.

Ke says China’s manufacturing supply chain offers many motor and robot-design options, while slower hardware iteration elsewhere limits the choices teams can explore. Hardware is still an open problem, even though a robot can be assembled and made to move.

Ke says hardware iteration speed matters, but building a machine is not the same as solving the software problem. Researchers still have to discover which design and algorithms produce useful behavior.

Ke says many of Pi’s papers are public, while Google and Tesla may disclose less. Even when research is public, a competitor still needs enough data and compute to train a model on the scale of Gemini.

Ke says Pi’s reading group discusses Chinese robotics demonstrations and may invite locomotion experts to explain work outside the team’s focus. Researchers look for ideas they can learn from without pretending to know every field.

Ke describes locomotion ranging from walking to backflips and dancing. She says reinforcement learning has been important in achieving such behaviors and asks whether the United States invests enough in robot hardware.

Ke says U.S. hardware still has room to improve and that Tesla keeps many details of its robot hardware confidential. She jokes that Tesla’s Shanghai factory is known for high output, underscoring the importance of manufacturing capacity.

Ke connects hardware gaps to education and career pathways: when an industry lacks training routes, fewer people develop the necessary skills. Immigration may help fill some gaps, but rebuilding an industrial base takes time.

Asked whether robots could become conscious, Ke says she wants to understand how consciousness might arise from physical processes such as biology, physics, and chemistry. She is unsure whether human-like machine consciousness is possible, though she sees it as rich material for fiction.

Ke returns to the idea of a robot ‘race’: robots that can assemble and repair themselves. She sees self-repair and replaceable parts as a milestone that could give robots a more flexible form of existence.

Ke says one Pi robot has been revised so extensively that its only link to the original may be its serial number; every component has changed. Xiaojun asks whether it should receive a new name.

Ke says this raises an ethical question: if every part of a robot has been replaced, is it still the same machine? She jokes about an ethics committee checking whether every component was really changed.

Ke imagines that a future of abundant productivity could leave people free to pursue very different lives. Human beings have only a brief place in history, she says, but they may still create something that matters to others.

Ke says a learned system may be difficult to interpret internally, yet its behavior remains bounded by the data and objectives used to train it. In that sense, even a black-box model can be described mathematically.

Ke sees productivity as an important human pursuit, though her robotics work can feel far removed from the humanities. She balances it by talking with friends who write fiction and make art.

Asked what might happen after people create agents, Ke says she does not know. She imagines robots developing their own aesthetic preferences, perhaps finding a QR code beautiful, and wonders what they might do if they became conscious.

Ke says she has never stopped imagining novels, though procrastination often keeps her from writing them down. She would like to create a story or game that reflects her tastes and share it with readers.

Ke imagines robots taking over unpleasant chores, leaving people more time for creative work. If robots could handle repairs and other specialized tasks, someone might even restore an abandoned castle and live there.

She says today’s research often depends on finding a specialist with the right insight. In the future, widely available general-purpose systems could make expertise in many specific tasks accessible to more people.

Ke writes two broad kinds of science fiction: stories set in very different worlds that explore human relationships, and stories that bring extraordinary or fantastical ideas into modern life.

Ke describes a story about someone who enters a future society and tries to change its social order. Though such a scenario feels distant, it lets her explore a person’s struggle, choices, and heroic journey.

The character wants to resist the strong social shaping that begins in childhood and create a society where people can choose more freely. Ke is interested in how the character gathers allies and persuades others to join the effort.

Ke says she enjoys bringing two very different worlds together to reveal shared human concerns. She compares this with captaining teams in online games, where she had to organize players, coordinate communication, and encourage the group when progress stalled.

Asked what motivates her, Ke says she is happy to be doing the work she wants to do. She does not need a fixed goal for every step; like Steve Jobs’s idea of connecting the dots, some meaning becomes visible only in retrospect.

Ke hopes to pursue projects in the next five to ten years that she would not have dared to imagine earlier. She is also open to practical work as technology develops, and says there is no shortage of things she would like to try.

Xiaojun asks each guest to recommend a book that had a lasting effect on them. Ke chooses Pierre Bourdieu’s Distinction, a study of social class and taste.

Ke says Distinction helped her ask how much her preferences are truly personal and how much society has shaped them. She sees Silicon Valley’s worship of productivity as another expression of social values.

Ke observes that Silicon Valley’s casual clothes, emphasis on simplicity, and focus on creation all reflect a particular value system. Even the rejection of older markers of status is a form of social taste.

The pair compare greedy algorithms, which choose what seems best at each moment, with dynamic programming, which considers the global result. Ke says a locally optimal choice does not always lead to the best long-term outcome.

Ke says none of us has a complete view of the future, so optimizing every step is impossible. She is comfortable doing nine things well and leaving one imperfect, and sees earlier unfinished paths as part of how she arrived at robotics.

Asked to name a favorite place, Ke chooses Edinburgh, where she studied for half a year. Its preserved historic buildings and distinctive physical setting made a lasting impression on her.

Ke recalls learning in college that a common belief about the hymen and first-time bleeding is medically inaccurate. The discussion prompts her and her friends to recognize how little they had been taught about their own anatomy.

Ke says the conversation with her friends exposed gaps in their basic knowledge of the body. Xiaojun then asks which research papers have most influenced robotics.

Ke says there are too many influential papers to name just one. She mentions diffusion policy, the use of Transformer architectures such as ACT, and earlier work in imitation learning and reinforcement learning.

Ke says robotics may enter homes in many forms, not necessarily as a single product. She expects continued exploration across different configurations and methods.

The final exchange turns to the possibility that our senses could be receiving signals from a simulator. Ke says language models’ limits show why describing a glass is different from understanding its physical properties.

Ke and Xiaojun end by considering idealism: if all experience comes through sensory signals, perhaps the world is mediated by a source we cannot directly inspect. They leave the thought as an open philosophical question.