99% of Factory Robots Are Dumb. Covariant Wants the Smart Ones Working in Warehouses First

Peter Chen joined No Priors as a frequent listener who said the show’s name left him no choice. He is co-founder and CEO of Covariant, the company building AI for robot arms. Before that he was a research scientist at OpenAI and a researcher at Berkeley’s BAIR lab, with a UC Berkeley PhD and a publication record in reinforcement learning, meta-learning, and unsupervised learning. Host Sarah Guo wanted priors in a landscape that still has few of them.

Two research obsessions pulled him into robots. The first was unsupervised learning from vast text, image, and video corpora, the work now branded generative AI: if you want capable models, you need a lot of data, and unlabeled corpora are where that data lives. The second was reinforcement learning, not just understanding the world but making decisions by trial and error. Robotics, he argues, is the combination. A robot cannot be a passive reader of text or pixels. It has to act, and the consequences are real.

That is also why he thinks robotics will push AI itself forward. Language models train on the internet. Robot models train on grounded, embodied interaction. The physical loop is the missing dataset, and collecting it is the reason he left research labs for a company.

The usual PhD-founder story is a lab technology mature enough to commercialize. Chen says that was not Covariant’s path. When they left OpenAI in 2017, AI was not good enough to make robots commercially useful. The insight was about the shape of the future, not a finished artifact. Foundation models that are multitask, data-hungry, and therefore more generalizable would transfer across tasks and make every task stronger. Language later proved the point. They bet the same would be true in the physical world.

The obstacle was data. There is no public corpus of robot actions and their outcomes. To build a robotics foundation model you have to build a company that can collect that data, and the only way to collect enough is fleets that already create customer value in production. Lab collection has a ceiling. Production data is a different order of magnitude. Chen calls it the Tesla approach: ship a car people want, attach a good-enough Autopilot, let customers create value, and let the data flow back.

Guo had seen the other sequence. In an earlier role she was involved early with Aurora and Nuro, and she is a personal investor in Kodiak, companies that tried to build the brain first. She now finds collect-as-you-go more compelling on cost. Chen agrees it has to be incremental. Each generation of research spend must justify a product you can ship; customers produce data; data trains the next model. OpenAI, Anthropic, and Cohere are not fully general yet either, he notes, but they are good enough to productionize a large slice of problems and buy the next increment. You cannot jump to physical AGI on day one. Real products keep the roadmap honest.

Six- and seven-axis arms are everywhere: flexible, precise, fast, durable, and inexpensive. Chen’s statistic is the one that matters. More than 99% of robots deployed in the world are “dumb robots,” pre-programmed to repeat the same motion, unable to adapt, talk to a person, or change the job on the fly. Covariant is not trying to AI-program those old, rigid use cases. The goal is orders of magnitude more applications, the ones that require intelligence because the scene keeps changing.

He offers a contrast. A Tesla factory robot that hurls a multi-ton car body with millimeter repeatability is an engineering marvel, and it does the same thing forever. An e-commerce warehouse robot faces hundreds of thousands of unique items that must be distinguished, picked, and packed without damage. Foundation models, in his telling, lift robotics from repetition to diversity. Covariant started in logistics because e-commerce dumped complexity into the supply chain while demographics and immigration made fewer people willing to drive ninety minutes to a suburb and work until midnight. The warehouses they serve often have more than 100% year-over-year turnover. Traditional programmed machines cannot pick, pack, and ship at the required reliability.

The concrete example is the put wall, a fulfillment step that sorts mixed orders. If Guo and Chen each ordered a new iPhone, a robot on the line has to pick one up without damaging it, identify it, and route it to Sarah or to Peter. Chen’s analogy: a network router forwards digital packets; a put wall is a physical router for goods. Identification and routing are often treated as solved, because conveyors and barcode scanners turn them into mechanical problems, and mechanical problems have answers. Grasping is the AI-heavy piece. Once a flexible model is in the loop, though, even the “solved” steps are worth redesigning. An AI can decide where the barcode is, or read the packaging the way a person infers contents without scanning at all.

Near-term, the company is staying inside warehouse manipulation because apparel, cosmetics, and meal-prep sites demand different skills and data, but the Covariant Brain is built to generalize, including onto humanoids. A working humanoid would be a universal chassis for a world designed around human bodies, and a product like Tesla Optimus would accelerate deployment; Chen is not waiting on it, because industrial arms already support a scaling business. Five years in, Covariant is about 200 people, with customers split roughly half Europe and half North America and robots in more than ten countries across three continents. Large retailers and e-commerce brands run the sites, and every arm feeds one foundation model, so what one robot learns makes the center stronger.

Guo asked him to define grounding. Read only internet text and you learn symbolic associations: “apple is delicious,” so when asked for something tasty you say apple. You still do not know what an apple looks like. Grounding pins those symbols to the physical world. Multimodal models such as GPT-4V already do a first version of this, trained on image-text pairs that occur naturally online. See enough Instagram photos tagged apple and the association forms. That is how ordinary multimodal understanding happens, usually via an image of the real world.

Robots still need two things that captions and YouTube do not provide. The first is precision. Natural language describes high-level concepts, not the submillimeter shape of a particular apple. A two-centimeter error on the boundary and a gripper closes on air. That precise physical understanding simply does not occur on the internet, and Chen calls it the first departure of robotics foundation models from general multimodal ones. The second is action-outcome pairs. How hard should you press a tomato? If you move an item too fast, does it bruise? There are not many robots doing interesting work, so those datasets barely exist. Human videos on YouTube are the closest proxy, and you still cannot read the newtons in someone’s fingers. The feedback loop has to be collected in production.

Do scaling laws apply, and can you predict how many robots buy the next jump in skill? The technical version, Chen says, already holds: more data, more model capacity, more compute, lower training loss, as in every other domain. What people usually mean is the looser version, the GPT-3-to-GPT-4 or Claude-1-to-Claude-2 step change in reliability and generalization. He sees some of that, and does not bet the company on it. A fully general model such as a hypothetical GPT-5 must gamble on out-of-distribution scenes it never trained on, which is the strong form of scaling. A domain-specific manipulation model does not. Coverage can be dense enough that the test case is already a training case. Get the domain data and you can fit it. Whether robotics scales, in his framing, is the simpler bet of whether you can obtain that data.

Guo pressed on whether Covariant is a scientific insight or a full-stack execution play. Chen’s answer is that the architecture has changed maybe five times, a significant rewrite every year, because you cannot marry a single design in a field moving this fast. The unique bet is one sentence: the future of robotics will be built by whoever has the most robotics data, and the whole company is organized around that thesis. The opposing philosophy is to rely on simulation and skip most real-world data; Covariant uses simulation as augmentation, not as a replacement.

Simulation, he argues, is a different problem in driving than in manipulation. Self-driving simulation is mostly about avoiding contact; two cars touching is a disaster. Manipulation that never makes contact has done no work, and contact with deformable objects is brutally hard to simulate. There is also a catalog problem. A typical customer warehouse may hold 100,000 distinct SKUs. Fully recreating that in a simulator can be more work than teaching a system on the real world. What he does believe in is a learned world model: train on reality, then imagine what would happen if you acted differently. That is not classical simulation, which hard-codes physics and lets an agent play inside the encoded rules.

Chen has described the industry as pre-ChatGPT. The moment, for him, has two requirements. First, generality of the ChatGPT kind: drop a robot into an arbitrary new scene and it learns to cope quickly. Second, high reliability. You cannot accept a robot that succeeds 70% of the time if the remaining 30% includes catastrophic outcomes. That makes the bar higher than chat. Solving generality and reliability together is why you need large amounts of high-quality data densely covering the target field. Hardware is the other half. Until humanoids are widespread, he expects the moment to show up in industrial settings first, because a cell that runs 24/7 can justify the body. A home robot used two hours a week cannot.

The warehouse of the near future is not lights-out. It is robot-augmented: one person overseeing 10, 20, or 30 machines, a physical copilot that multiplies a single worker. He thinks that pattern, AI amplifying one person rather than deleting people, will describe most AI domains, not just robotics. If he has to guess at the first intelligent robot an average consumer meets, it is a home unit without much manipulation, something that follows you, talks, and navigates, closer to a Roomba or Amazon’s two-wheeled Astro than to a housekeeper with hands. Navigation clears the technical bar before grasping does.

Safety, for Covariant, is almost a category error in the industrial present. Factory robots already live under mandatory rules. Handwritten programs can be dangerous too; you can already instruct an arm to do harm. The industry’s answer is cages, or certified controllers if there is no cage. Covariant deploys under the same rules, so the robots are safe by construction. That is a different problem from hooking an arbitrarily expressive agent to a home robot, which he compares to giving a language agent unrestricted Python and internet access. That is where alignment research belongs. It is not the near-term industrial case.

Looking forward, outside robotics, he expects at least last year’s magnitude of progress again, maybe more. Peel back LLMs and image generators and the methods still look primitive. Language models are mostly next-token prediction, which reinforcement-learning people call behavior cloning: ask the AI to copy another agent. Mimicry has a ceiling. A pile of proven RL tools has not been deployed yet, which is why he calls further progress guaranteed. He is equally excited that the open-source movement keeps putting those advances in a broader community’s hands.

Guo’s last question was his favorite robot science fiction, realistic or not. It is not a book. He likes Westworld, a series about machines learning to decide. For a founder whose days are spent teaching arms to grasp without crushing the tomato, the choice is on-theme. The practical filter he leaves behind is simpler than the show: look for production robots, a single improving model, dense data rather than hoped-for emergence, and jobs people already refuse to keep.