Judge a robot by the environment it forces to change, not by the demo

Lex Fridman opens by asking Rodney Brooks for the most beautiful robot he has worked with. The answer is Domo, built by graduate student Aaron Edsinger with mechanical engineer Jeff Weber, now sitting in Daniela Rus’s office at MIT CSAIL. It is an upper-torso humanoid: two arms, three-fingered hands, series elastic actuators, cable drive, motors hidden in the body. Edsinger did not submit a budget or a plan. He started spending, spent a lot, and anodized parts in different colors until the machine looked less like an engineering product than a work of art.

The face is only eyeballs, and those eyeballs move, with cameras and a visual attention system that turns toward people who walk in. Brooks has a rule he repeats to students: when you make a robot, its appearance is a promise about how well it will interact. Do not overpromise. The joke, which he expects Fridman to get, is that if your robot looks like Albert Einstein it should be as smart as Albert Einstein. Domo can look and pay attention, so that is all the face is allowed to do. Japanese androids that look exactly like people are the opposite of the principle.

Humans and dogs do not use eyes only for attention. A dog looks at you, looks at an object, and looks back, designating a shared target. Intent works the same way on the factory floor. Baxter and Sawyer, the collaborative arms from Rethink Robotics, had a screen of graphic eyes that were not the cameras. The eyes looked in the direction the arm was about to move, so people standing nearby were not startled. Later in the conversation Brooks returns to those eyes: some customers hated them and got a switch that showed graphs instead; others missed the more human version. What disappoints him is that many of the best roboticists still treat the machine as a machine, and refuse the human connection.

Brooks was born at the end of 1954 in Adelaide, South Australia. Two How and Why Wonder Books dated 1961 — one on electricity, one on giant brains and robots — taught him simple circuits and binary when he was eight or nine. The pictures were not real robots: nuclear-handling arms, underwater welders that looked like people. He tried to build them for the rest of his childhood and learned his mechanical limits early, so he built brains instead. The first learning system was chemical: an ice-cube tray, each well a cell, nails as electrodes. Voltage grew a copper bridge; over time the network learned to add. Budget decided everything — nails and a tray.

Transistors came later, then gates and flip-flops. At twelve he built a tic-tac-toe machine from old telephone switchboard switches that always won: nine three-position switches, empty or X or O, a light bulb for the next move, no active components. As a child he liked machines that could beat adults at a game the adults could not beat. Fridman hears the rebellion in it. Brooks still lists that twelve-year-old circuit among the things he is proudest of. People standing nearby said there was magic in it. He agrees there was magic. He also insists each next step is a hard-fought battle, and seeing the magic is not a reason to assume the curve continues.

Fridman quotes the opening of Turing’s 1950 paper — can machines think? — and asks whether the ice-cube tray might one day think. Brooks’s answer is immediate. Of course machines can think, because you are a machine and he is a machine, and both of them think. Any other philosophical position strikes him as a little ludicrous. What “think” means, if it is not what we do, is unclear. Whether we have a clue how to *build* such machines, and whether we are smart enough, is a different question. Maybe we are not.

Brooks is halfway through a book of about 480 pages, working title *Not Even Wrong*. The first thrust is the history of what we call computation, from Latin manuscripts by Napier and Kepler in 1614 and 1620 through Babbage and Lovelace to Turing’s 1936 paper. That paper did not set out to invent computation. It set out to negatively answer one of Hilbert’s later problems — disproof being easier than proof, Gödel having already handled the other two. Turing’s Cambridge professor had recast the question as whether there was a mechanical process. The people then called computers sat with paper and rules, writing numbers, iterating toward more significant digits for engineering tables. Turing needed a machine that could do that: infinite paper, a tape, instructions a person could carry out with pencil and paper. He showed the system could not do what Hilbert wanted, and admitted in the paper that he had no real argument it exhausted everything computable — only intuition.

From 1936 into the 1970s people still struggled with whether that *was* computation. Marvin Minsky’s *Finite and Infinite Machines* shrugs that wood or plastic does not matter. Donald Knuth, around 1968, says computation is the stuff Turing described, steps a person could do without too much trouble — knowing whether Fermat’s Last Theorem was true would be too much trouble. By 1975 the standard textbooks were saying intuition says this is about right, and it happens to be easy to implement in silicon. Then Moore’s Law took off. Unlike mass or charge, computation is not given by the universe. It is a name we chose, modeled on humans with limited memory, that turned out to have a spectacular implementation.

The second thrust is a two-by-two matrix: science versus engineering, intelligence versus life. That yields AI, artificial life, neuroscience, and abiogenesis. All four took their current form between 1945 and 1965, with overlapping players, many of them around Warren McCulloch at MIT’s Research Laboratory of Electronics. McCulloch and Pitts’s 1943 paper, inspired by Bertrand Russell, modeled neurons as logic gates with no real evidence. Von Neumann’s 1945 EDVAC design had three references; one was that paper. Minsky’s 1953 dissertation was a neural network. Brooks titles two chapters “Wait, computers are people?” and “Wait, people are computers?” Three of the four fields then took computation as their primary metaphor. When Elon Musk’s rocket goes up, he asks, is it computing its way into orbit? The word has been asked to do work it was never meant to do.

Turing-style computation, he argues, fits hunter-gatherer thinking: places, containers, stuff that stays until someone changes it — place cells in the hippocampus plus cortex. Outside that metaphor range we invent tools such as calculus. We still demand physical metaphors, which is why quantum mechanics has been a hundred-year argument over particle versus wave. Brooks’s bet is that in a century or two neither quantum mechanics nor dark matter will be discussed in today’s terms, the way phlogiston went away once burning turned out to be oxygen outside the matter, not stuff inside it. A drum knows its stationary points without computing them. The first program Ada Lovelace wrote computed Bernoulli numbers, which are exactly what you need for those points — and it had a bug, arguments to a divide reversed, and never ran, because the Analytical Engine was never built.

Every intelligence we know — dogs, octopuses, us — perceives the world and acts in it, and the perception of objects is phenomenally better than a snapshot. The box between Brooks and Fridman looks blue, but blueness is color constancy, not a property of the photons. That is why a few pieces of tape on a stop sign fool a deep-learning system: redness is not in the photons either. Dogs smell a lamppost and recover how many dogs visited in the last ten hours and when. We take a single image and call it perception. Philosopher Brian Cantwell Smith’s “registration” — a correspondence between you and the object — is squirmy and, Brooks thinks, absent from our systems. When deep learning started labeling images, people said the symbol-grounding problem was solved. He says the labeling problem was solved, at some percentage accuracy. That is a different thing.

He shared an office with Hans Moravec when Brooks was on computer vision and Moravec was on a first mobile robot. He does not treat Moravec’s paradox as a paradox. Evolution spent about 600 million years of multicellular life on perceiving and moving; hunting and gathering came much later; language maybe a couple of hundred thousand years; agriculture ten thousand. The late, thin layers sit on a stack that took ages. The poor intuition about difficulty has a social origin: the late-1950s MIT AI lab was full of kids who had gotten into MIT by being good at chess and integrals. That was “hard.” A baby could see. Seeing was not intelligence. Chess kept the crown until Deep Blue, then the same newspaper stories ran for AlphaZero.

Asked which category of robotics problem he would solve instantly, he does not pick. Then he tells a grandfather story. His sixteen-month-old grandson, first time in a new house, walked to a low window he had never reached, one hand on the frame, the other turning a handle he had never seen, and opened it. Two different hands, two different actions, a mechanism never encountered. At eighteen months, given a box of locks, the child did not paw every surface. He could see where the mechanism was and explored those. Reinforcement, yes, but behind a pre-filter that cut the search space. At DeepMind in London, behind a door other Google employees cannot enter, researchers described letting robots try everything until reinforcement learning figured it out. The robots were Sawyers Rethink had sold them, chosen because they are compliant and stop when they hit a wall. Brooks asked to see them. No: secret. They were his robots.

Fridman pushes on beauty in the deep-learning wave. Brooks grants the shock of ImageNet-era computer vision — “wow” — and refuses the leap to having solved vision for robots. Self-play and AlphaZero, he notes, are already in Donald Michie’s 1961 paper, which introduced reinforcement learning; Arthur Samuel’s checkers program in the early 1960s beat a world champion using a learning mechanism Claude Shannon had sketched, run on IBM’s production-line burn-in computers, more compute than anyone else in the world. Fridman wants the realized promise, the reusable-rocket version of an old idea. Brooks’s counter is that chess is easy and our view of ourselves is overblown. A 1946 report pitted an abacus against an electronic calculator and the human won. No one is surprised now that a machine does a billion floating-point operations a second.

“Learning,” he says, is a suitcase word and a deadly sin when used too broadly. Learning a city, chess, Latin, and a bicycle are different capabilities. Press offices now announce that company X leads the world because a system got better at a thing, under a measure, under a set of conditions. Most of these systems fail when the conditions change a little, in a way humans do not. After AlphaGo he asked DeepMind what would happen on a 21×21 board. Total failure, they said; a human would at least be able to play. He notes they have since published algorithms that stretch. He still will not call that intelligence. Perfect-information games have a short description of state. That is not registration and perception in the world.

AlphaFold is the surprise he will own. Getting so many protein structures right on a famously hard problem startled him. Usefulness was limited because you did not know which predictions were right; a later tool was trying to fix that, and might actually help structural biology. Fridman offers GPT-3, AlphaZero, AlphaFold as candidates for a core of something. Brooks separates surprise from usefulness and still will not say intelligence is solved. Every few years someone tells him he does not understand how powerful the new thing is. He has a track record of the world doing this, bigger computers in the 1990s included. Seeing magic and assuming the next step is automatic is the mistake he wants people not to make.

The conversation turns to cars. Brooks likes the 2021 Mercedes E450 he arrived in: lane keeping, blind-spot warnings, a metrical top-down parking view stitched from cameras. It is the longest he has owned a car without digging into it. Useful technology is not the argument. The argument is that people see a demo and interpolate that we are almost there. A car drove over 55 mph for more than ten miles on a public freeway with traffic, fully autonomously, in Munich in 1987. Coast-to-coast hands-off claims go back to 1995 and were not a hundred percent. We have been working on this for 35 years.

He will admit surprises. Sitting in Mobileye’s implementation, his model of what classical computer vision could do in a real car was too pessimistic. Tesla throwing Mobileye out and catching up from scratch, including a data engine that splits driving into tasks and mines the fleet for edge cases, made him rethink what engineering can cover — a point Fridman, citing Andrej Karpathy’s involvement, presses. Brooks still doubts vision-only without cameras that approach the human eye’s dynamic range. The bad cases are sun on white, glare a person would handle. He will not say it just works.

About 35,000 people a year die in U.S. motor-vehicle accidents. Engineers tell him that cutting deaths by 10 percent would be a triumph. His prediction is the opposite: if autonomous vehicles kill more than about ten people a year, the public will scream, even with 35,000 human-driven deaths. The expectations are a different set, and they are not rational. New technology also changes the rules of the game. When cars entered daily life, cities were rebuilt and laws rewritten. Jaywalking was not an offense until car companies pushed it, to get people off the road. Large adoption will not be “take out the driver and keep the same car doing the same stuff.” The tail of cases is too long.

How many fully autonomous train systems are there in the United States? Fifteen or sixteen, most in airports. The first urban mass-transit one was expected in Honolulu in 2017; delayed, but it will get there. Brooks has ridden a fully autonomous line out to the artificial island in Tokyo Bay. What you see is engineering of the environment: a double set of doors at every station, train door and platform door, a track people cannot climb onto, even where the line runs among buildings. That is what makes the system acceptable. He thought, wrongly, that the leftmost lane of Highway 101 would become a dedicated autonomous lane five years earlier. It did not. He suspects Tesla calling the system Full Self-Driving blocked the collaborative infrastructure path. Long-haul trucking at night on freeways might make sense; getting on and off still needs infrastructure. His gut is that things speed up when the environment is engineered, because that is what happened with every other technology.

On Highway 280, mostly concrete rather than asphalt, Brooks has noticed that the white lane lines now have black borders. His car’s lane-drift system would not work without them. He does not know whether the paint was changed to help the systems, but it is an instance of infrastructure quietly following technology. Waymo running without a driver in Chandler, Arizona, was another surprise he will grant — watching the wheel turn with nobody in the seat. As of late the previous year they were aiming at hundreds of rides a week, not thousands, with safety people still in the loop at some ratio, rescue vehicles for cars trapped by a cone. The interpolation he wants to block is “it’s here, it’s done.”

On 1 January 2018 he published a set of dated predictions and promised to check them every New Year’s Day through 2050, when he would be in his mid-nineties. Everyone else makes forecasts, they fail, and everyone forgets; he wanted a scoreboard. The topics are AI, autonomous vehicles, and space. He is not adding new predictions, only scoring this set. A dedicated self-driving lane on 101 was too optimistic. Driverless taxis on every street in Cambridgeport, Massachusetts, by 2035 now looks optimistic too: narrow roads, one-ways, people driving the wrong way. Internally he has grown more pessimistic on some of these.

North of Market in San Francisco, wide grid streets, the main complication being delivery trucks, a taxi system with designated pickups is still a few years away. South of Market, much longer, unless the roads are re-engineered. He lives there and sees the test cars daily — Waymo constantly, Cruise, Zoox less often — always with a driver. Sometimes the driver has no authority to take over without the home office; they sit and wait, and incident-resolution time tells you more than the press release. One Friday around 8 p.m., a Cruise came down the hill, turned right, and stopped dead covering the crosswalk because a woman stood two feet away unlocking her car with a fob. Brooks could see she was not going to jump out. The safety driver did not take over. Then a man with a stroller found the crosswalk blocked, went behind the car, off the crosswalk, into the intersection, and around. No human driver would have sat there that long. Safety was compromised for people who did not sign up for the experiment.

He is generally not a fan of anecdotes, and says so. The structural point is who is affected versus who decides. Where it *will* happen first is campuses, gated communities, airports: other people who know the vehicles are there, speeds at which it is always safe to stop dead. Those vehicles may not look like cars. The experience of sitting in a machine that waits is miserable for most riders and transformative for someone in their eighties in a retirement village whose kids have taken the keys. That value is orders of magnitude smaller than “the world’s roads.” Robots that deal with people, he believes, must learn to push back. A colleague tortures autonomous cars by putting a foot off the curb. Teenagers will do it every day once they learn it works. Waymo, before the name change, found that at four-way stops it had to nudge forward or other drivers would beat it forever. A robot you can bully is a bad experience.

Fridman presses on harshness. Believing the impossible is required — 1903 flight to 1969 on the moon is 66 years, Brooks’s own lifespan. Brooks agrees optimism matters. The harshness is for people who are not building, who claim a demo means it will just happen, and for a different argument with Elon Musk: SpaceX reused demonstrated pieces (vertical landing, grid fins since the 1960s, an insurance industry); Hyperloop had not. Musk, in Brooks’s view, does not distinguish ready technology from mouthing off, and other people then compete to do the undemonstrated thing. Fridman draws a line at lying versus sincere belief. Brooks’s frustration is leaps that tell him he does not understand, from someone who co-founded iRobot.

What is he proudest of at iRobot? In March 2011, while he was still on the board, robots went to Japan to help shut down Fukushima Daiichi. He visited in 2014; some of the robots were still there. They could do it because about 6,500 iRobot machines had already been deployed in Iraq and Afghanistan, teleoperated with onboard intelligence, dealing with roadside bombs — nine years of harsh field experience. Japanese robots were collecting the hype: look at the Honda walker, the future is here. They had not been deployed. They could not do a thing. “The robots that got all the applause,” the Chinese article records him saying in spirit, “couldn’t do anything.”

Then the simple one: Roomba, tens of millions of homes. He jokes that he started as a pure mathematician and turned into a vacuum-cleaner salesman; entrepreneurs should be ready. A lawsuit years later sent him through 1990s email and reminded him of the joy. The target shelf price was $200. Electrolux’s Trilobite sold for 2,000 euros, which they did not consider a consumer product. That price meant the bill of materials had to be brutal. After MIT meetings in Taipei he would stay extra days, go to Hsinchu, and visit tiny companies around TSMC that could innovate without a fab, stripping a Motorola 6802 — the Apple I chip — of half its silicon and still having it run. In Hong Kong he hunted handheld-game and greeting-card chips because the compute budget was about fifty cents. Interrupt handling was often too weak. He found a Winbond part with 512 bytes of RAM, in budget, with the capabilities they needed. Emails from that period are just “I found this.”

They did not imagine the company would get so big. Fourteen failed business models until 2002, then two winners in the same year. The board authorized a Christmas 2002 build; they went three times over and built 70,000, released 18 September, sold out by Christmas. They were gutsy. They did not think it would take over the world. He stayed another nine years looking for the next home robot. Almost twenty years later, the primary product is still Roomba.

Why is a successful robotics company so hard? Brooks names three pits he has stood in. Founders’ expectations of capability are off — the founders’, not the customers’. Pricing is wrong; what a customer thinks is a valid price is not necessarily rational. And getting people to adopt a new technology is just hard. He has suffered from all three, with more failed companies than successes. Asked whether a trillion-dollar robot company will exist, he will not predict: he is still starting robot companies, and nobody in the 1990s thought Apple would be a trillion-dollar company. He does see market pull in the home from demographic inversion — too many older people, not enough caregivers — but the robots are not yet capable enough for the economics. Garage-to-scale is very hard without real capital; he mentions an $80 million Series B for Covariant, and $150 million lost at Rethink.

Rethink is the failure that still hurts, and the pride is specific: they got robots out of the cage. People and arms could stand next to each other, safe, and humans could retask them without writing code. That pair of expectations is now advertised by companies large and small. He let himself be talked out of the original plan in a meeting he still remembers. The target was a $3,000 force-feedback robot, safe around people, sold not to factories that already had arms but to people who would never buy one. Plastic gearboxes, cheap enough to be opex not capex, replace when they wear. Series elastic actuators could measure force. Control engineers hated the torque ripple. A mechanical engineer said a metal spur gearbox would take six weeks. It took two years. Price went up. The CEO wanted two arms. Baxter shipped at $25,000.

At $25,000 the only buyers were factories that already had arms, cheaper for two than what they owned, and those buyers wanted 0.1 millimeter repeatability and familiar speeds. Brooks kept saying they did not need position repeatability: use force control like a human. To insert a part in a right-angle slot, position control fixtures the world and goes there; force control slides until it feels the surface and mates. Humans get precision the second way. Customers asked why every other robot had the spec. So they spent two more years on a $35,000 single-arm machine with a gearbox that itself had torque ripple — the problem they had tried to avoid. Then the death was not technical. Buyers did due diligence; every consortium had Chinese capital; CFIUS, historically invoked twice, was enough for bankers to say it would not pass. Investors walked. Two buyers remained as cash ran out. One used heavy legal pressure, promised more, dropped out when the company was broke, and bought the assets at a thirtieth of the prior week’s offer. “It was a tough week.” He said he would never start another company. The salve was that three firms hired the whole team within a week; some people did not even change desks, because a buyer rented the space.

Will humans fall in love with an AI that loves them back? People fell in love with cars in the 1950s, Brooks says, which is a different love. A lifelong partnership you grow with is a long, long way off; *Blade Runner* had the timescale totally wrong. The hard part is again Moravec: a form that perceives and can be perceived. Voice only, *Her* or Alexa, might not be far for some people. Science fiction’s method is today’s world with one thing changed. Reality grows together. The movie he thinks gets this right, however schmaltzy, is *Bicentennial Man*: as the robot tries to become more human, humans put the robot’s technology into their own bodies. We already modify babies and stuff devices into ourselves. He has hardware in his ears so he can hear Fridman. Future people will not be today’s people.

Joseph Weizenbaum was shocked in the 1960s at how much people would confess to ELIZA. What is missing if you want someone to talk for an hour, the way Amazon’s Alexa Prize frames success as a twenty-minute conversation, is continuity. You cannot refer to yesterday. There is no ongoing self, no shared moments that last in memory, no intention — not even “wanting to be happy.” Brooks is not saying it cannot be done. He is saying it is a minimum requirement, and nobody has shown what it feels like. Fridman thinks that bar is easier than solving intelligence and nearer-term. Brooks will not grant the drink with Alexa yet.

Read Turing’s paper, he says, and Turing is not offering a good test. It is a rhetorical device: if you cannot tell the difference, you must say the computer is thinking. It became a weird game of fooling people. In the late 1980s the MIT AI lab’s AI Olympics included the original imitation game — man or woman on the other side of the teletype. One man found a dead-giveaway question: did you have green plastic toy soldiers as a kid, and what did you do with them? A woman pretending to be a man would talk about lining them up and fighting wars. A man said he stomped on them. That, Brooks says, is what the Turing test with computers has become: hunt the trick question. Conversation *not* posed as a test is the actually hard problem, more intuitive than computer vision as a measure of distance, because language is so human that the gap is visible.

Researching the book, Brooks’s disappointment is in himself. The people active in the 1950s and 1960s he mostly knew when they were older. MIT had Thursday faculty lunches. He sat next to them and did not ask. A lot of motivations are lost, permanently. He would have asked J.C.R. Licklider, who envisioned human-computer symbiosis, pushed it at DARPA, and funded Project MAC at MIT in 1963, what counted as success, failure, and progress. He would have asked Minsky why he and Seymour Papert came down so hard on neural networks in *Perceptrons* in 1969, with theorems that were all negative, after Minsky’s own PhD had been on neural nets. Did Minsky know the book would freeze the field for a decade? Brooks suspects not. Minsky once said in an interview that the key to staying productive is to hate everything you have ever done. Maybe that explains the book.

Google buying eight or nine robotics companies was, on net, bad for the field: people no longer had to make a company succeed, were locked away for years, then the line was shut down. Some firms pay fabulous salaries and you never hear from the people again — no public talks, collected like movie stars not allowed to make movies. The big-founder dream at a place like Google may not be well founded and may not have the impact its believer thinks. Brooks still grants that a big dream is a unifier: bright minds around a dream spill adjacent ideas, and that is how a lot of progress happens. His 1 January review of 2020 said the year would not be remembered as a watershed for machine learning. Nothing surprising happened. Paper counts exploded; the work was incremental. The shake was deep learning hitting ImageNet.

The joyful memories are access to the future before anyone else had seen it. Stanford in 1977: terminals with live video, digital sound, a Xerox graphics printer that could print arbitrary black-and-white pictures. MIT Lisp machines, the first personal computers, a hundred thousand dollars each; arrive early and you got one for the day. Email every day since 1977, even when the host field was eight bits. One night the AI lab mainframe died. Brooks and engineer Robert Paul pulled panels off a washing-machine-sized disk drive. Donald Knuth, very tall, working overnight on TeX, stood over them and asked when it would be fixed. They socketed a replacement 7400-series chip in backwards. Smoke. They were nervous because Knuth was watching. They got it running. That was 1978 or early 1979, before the lab left the building in October 1979.

Advice for young people: there is no one way. There is pressure to produce publishable papers. He came from an age when you could rebel against that and still succeed; it may be harder now. Do not get too caught up in what everyone else is doing. If you want real impact, be ready to fail a lot of times, which means a lot of unsafe decisions, kept up long enough that one of them works. Or you may end up with a lousy career. That is possible. There is no path of all-safe decisions that actually contributes.

Mortality: when COVID arrived, before anyone knew how bad it would be, he thought about it, and it made him work harder on the book. Then the new company went more than full-time and the book went on hold. He wants to finish it. What he fears is dribbling, losing it, the self gone. That the ride ends he has known for a long time. Knowing and knowing are different, and it feels closer. The prediction blog is a pushback: review for 32 years, into his mid-nineties, and every annual post is a step toward his own deadline. Legacy, if he gets a vote, is finishing the book so that one person reads it, thinks a little differently, and that leads to the next big thing — and a hundred years from now someone says on a podcast that the book changed everything.

The meaning of life: we are all bad at it. Immediate things are easy; the long things are not; on the big picture we bat poorly. Atheism tells him it is random. What he wants to understand, and what the book is about, is how order comes from disorder — why this little pocket of complexity called Earth showed up in a random universe, how common such pockets are, whether they last. He cannot believe we are alone; that would be too cruel. The colder answer is a graveyard of civilizations, and us too, and all of this forgotten, including the robots. Historically, close to a hundred percent of people are forgotten. He knows no one alive who remembers his great-grandparents. And still, despite the absurdity and the stretches of meaninglessness, life is pretty fun. For him, one of the most fun things is robots.