A Robot Is Whatever Still Has Magic

Kate Darling is a research scientist at the MIT Media Lab who studies human-robot interaction and robot ethics. This is her second conversation with Lex Fridman, coming after the publication of *The New Breed: What Our History with Animals Reveals About Our Future with Robots*. Fridman opens with the question she uses to start the book: what is a robot? He offers his own answer first. A robot, he says, is something with some intelligence and some magic, a shine in the eye that lets it navigate the uncertainty of life. On that definition, a self-driving car counts.

Darling likes the word magic. She does not define robot in the book, she says, because there is no definition everyone agrees on. Look backward through time and a pattern appears: people call things robots until the mystery wears off. A vending machine used to be a robot; now it is furniture. Roboticists prefer a technical checklist: physical embodiment, sensing, autonomous decision, action on the environment. That list sounds rigorous until you apply it to a smartphone, which can do all of those things and which almost no roboticist will call a robot.

The definition that actually worries her sits in ordinary people’s heads. Google Image Search for “robot” and you get a torso, a head, two arms, and two legs. That humanoid picture, she argues, is the one we need to get away from, because it trips up every conversation that follows. The rest of the episode is an attempt to pull the discussion off that default and onto a different set of comparisons.

Why does the humanoid form trip us up? Darling gives two reasons. The first is technical. Early AI researchers really did try to recreate human intelligence, and some still do; there is academic value in that. It is not, however, where the field ended up. AI does not think like people, yet public talk about intelligence still uses human intelligence as the only origin point on the map. The second reason is blunter. Recreating a skill we already have, she says, is boring. The scientific project of understanding ourselves is one thing. The practical project of using these systems is another, and that one gets interesting when you build a skill set we do not have and then partner with it.

The two standard arguments for humanoid robots are communication and infrastructure. People relate most to things like themselves, and the built world has stairs, narrow passages, and door handles. Darling grants the second. On the first, social robotics and human-robot interaction research keep showing something else: you only need enough social cues for someone to relate. The thing does not have to look human or even act human. R2-D2 beeps and boops; it is a trash can on wheels; people love it more than they love C-3PO, who actually looks like a person.

Fridman is struck by how little it takes. A glowing orb, a voice, a scrap of basic interaction can start anthropomorphism. Teams over-engineer this, both of them say. Simplicity often travels farther, and any animator already knows it. The people who designed Anki’s Cozmo, Fridman argues, should have been Disney people, not roboticists. Roboticists are strong inside their discipline and often clueless outside it.

Darling pushes the point into the century. Early-21st-century robotics, she says, will not be late-21st-century robotics. Call yourself a roboticist today and you are mostly doing controls and mechanisms. Call yourself one later and the unsolved problems will sit in the social layer, in perception, in the “brain,” not in basic control. Fridman treats that shift as robots coming out of the closet. They have been in factories for decades. Now they are entering grocery aisles, hospitals, and sidewalks, and companies keep making the same mistake: they spend everything on engineering so the machine does not hit a person, and they forget to teach it how to say it screwed up.

His line is that you should not try to solve perception and control in isolation. You should teach the robot to say “oh shit, I’m sorry, I fucked up,” or to ask for help, or to communicate uncertainty. Humans have not solved perception and control either, he notes; we are just very good at it. The magic is self-deprecating humor and self-awareness about where the flaws are. Darling agrees there is a whole body of human-robot interaction research on how to do this. A lot of the companies shipping robots do not do HRI at all. They do not have that person on the team.

Then Marty walks on. Stop & Shop, a supermarket chain in the northeastern United States, deployed a six-foot aisle robot that looks for spills on the floor. Fridman, unkindly, says it looks like a giant penis. Darling’s complaint is more structural. The company worked so hard to get a functional robot into the store that it never asked how people would react to sharing the aisle with it. People hate Marty, she says, and the hate is not about jobs.

Darling and an MIT student, Daniela de Paola, had planned a study of Marty just before the pandemic, then cancelled it because nobody wanted to go to the grocery store. The early observations were already enough. De Paola noticed that everyone on Facebook in her circles was complaining: what is this creepy robot, it is watching me, it is always in the way. A rough Twitter sentiment analysis found the biggest spike of negative mentions on the day Stop & Shop threw Marty a birthday party, with free cake and balloons. Who complains about free cake? People who hate Marty.

An online Mechanical Turk poll pointed at the same three grievances: it is a surveillance robot, it is watching, and it gets in the way. Almost nobody led with job loss. Marty’s actual job is looking for spills. It does not look at people. Darling suspects the two big googly eyes slapped on later made the watching feeling worse, and that nobody on the design side thought that through. Fridman offers the analogy that structures the next stretch of conversation: is Marty a real-life Clippy?

Clippy, the paper-clip assistant in Microsoft Office, is a machine Fridman likes and a great many people hated hard enough that Microsoft killed it. Clifford Nass, a Stanford figure in human-computer interaction, argued that people treated Clippy as a social agent rather than a tool. When a tool interrupts you and forgets what you just said, it is badly designed. When a social agent does the same thing, it is rude. The complaints were louder than a non-character piece of software would have drawn, and the same logic, Darling says, maps onto a six-foot robot blocking an aisle with a screaming child in the cart.

Fridman offers a marriage metaphor. A lot of people complain about marriage while secretly loving it; busting each other’s chops is part of closeness. Maybe Marty is not doomed. Maybe it just cannot talk back. Darling sees a wide design space for humor. The robot could lean in: yes, I am CIA, monitoring your every move. It could mock the shopper’s own paranoia: I am collecting your data because you are so important. Any self-deprecating move that names the fear beats silence. People already respond more extremely to robots and artificial agents than they do to ordinary machines, she says, because they view them as social agents, not objects. Love runs hotter. Hate does too. The middle is narrow.

Marty’s awkward position is that it is neither lovable nor gloriously hateable. It sits in the middle, looking like a corporation’s camera on wheels rather than a creature with its own task and personality. Fridman notices a subtle trust mechanism: the more personality you give the thing, the less it feels like it is piping data to someone else. When we talk to another human being, the default assumption is that the sentence is not going straight to the CIA. Facing a machine that is obviously an arm of a large company, that default collapses. Crude personification makes it worse, because people can see the PR department trying to put a friendly skin on a surveillance device.

That, for Fridman, is the same sour note he hears in Meta’s first metaverse avatars. Something about the way Mark Zuckerberg looks in that world does not feel genuine, and he cannot quite put it into words. The design problem is the social-robotics problem in another medium: how do you build a virtual space where connection has the texture of being in the room together? Darling has mostly seen the jokes on Twitter. She has a different question. Meta poured so much money into this. Why does it look like that?

Fridman has talked to John Carmack, then a part-time CTO at Oculus, about why the avatars landed so badly. Bureaucracy is part of the answer. Large corporations, he says, have a way of killing the independent artistic flame required to make something compelling. An idea runs through PR, legal, and a stack of “is this a risk” reviews, and everything interesting gets stripped because it could be controversial. What remains is generic. Independent game designers, and Grimes in a critical tweet, have already solved versions of this problem in small groups willing to be edgy. Clippy itself, he suspects, would not survive Microsoft or Meta today. There would be meetings about why it is not sufficiently friendly, and the artist who originated it would be gone.

Darling’s version of the same worry is the harm-finding department. Build a humanoid like Sophia and you inherit a list of real questions: are you reinforcing an unrealistic beauty standard, objectifying women, defaulting to white? Those questions are not fake. Her claim is that enough creativity can produce a non-humanoid form people relate to *more*, and then you never open the bias can. The organizational problem is a team whose job is to find harm in things, including things that have no harm. Put that team in the first round of creative review and the idea dies. Edginess, she insists, does not have to hurt people. Great humor does not have to be cruel. Risk-averse lawyers will still say no.

Fridman’s proposed fix is a benevolent dictator in the Steve Jobs or Jony Ive mold: someone who can cut through lawyers and PR and still has a good heart. Great designers, he says, imagine millions of people using the thing and feel joy at that picture. Darling agrees on the direction and is more careful about the method. You can keep the judgment. You still need a diverse set of people looking at the work, so they can show you your blind spots. Listening is separate from obeying. You can ignore what you do not believe is an issue. You have to hear it first.

Darling’s book includes a photograph of two hospital delivery robots. The caption, which nobody in PR edited because nobody edited the book, reads that their sexy-nurse names, Roxy and Lola, made her roll her eyes so hard they almost fell out. The form factor is fine, a little box on wheels. Naming itself is fine; research shows that a name helps adoption. When the machine errs, people say Betsy made a mistake, let’s help her, instead of this stupid robot does not work. The problem is *which* names. A large literature finds that robot names track gendered stereotypes of the job: helpful, assisting, nurse-like systems get female names and voices; all-wise decision systems like Watson get a booming male voice. Roxy and Lola bolt a particular gender fantasy onto nursing for no engineering reason.

That, she says, is a can of worms you can avoid at zero cost. People do not change the names because they are not thinking. Fridman asks whether designers should simply refuse to gender robots. Gender is a minefield, Darling answers, and it is very hard to stop people from gendering a machine anyway. Give it no name and an ambiguous voice and users will pick something. That might be better than entrenching a choice you decided was best, but attributes people can identify with still help on the engagement side. Fridman’s sharper charge is aimed at roboticists he knows who refuse to gender *or* name the machine, even refusing a name you could use in conversation. He calls that irresponsible. People will anthropomorphize. Skipping the choice only removes you from responsibility for how they do it.

Even a Boston Dynamics quadruped with no intended social component gets a social reading. The arm sticking up from the robot dog is an arm; everyone sees a head. The company has had to acknowledge that, especially after police departments started using the dogs and a public-relations nightmare followed, which is when they started hiring human-robot interaction people. Fridman adds a colder observation. Boston Dynamics makes its money in industrial inspection and dangerous work, not in the living room. A little public fear can be useful: the videos go more viral, the brand gets stronger, and they never promised companionship. Darling thinks the early videos were a labor of love, overseen in obsessive detail by then-CEO Marc Raibert, who would not allow a slope in the frame that gave the robot an unfair advantage. They take the social reading seriously now. She still prefers the engineer-made ones, because those were shot for fun.

Asked what robots are actually good at today, Darling’s first correction is that we are way behind where people think we are. Warehouses are not fully automated. Amazon’s then-new warehouse robot mattered to her because it was the first autonomous warehouse machine safe enough for people to be around. Most industrial robots strong enough to move something heavy can seriously hurt a person, which is why they used to run on fixed lines. The exciting shift is cobots: people and robots in the same space without anyone getting harmed. The hard part is not malevolence. It is knowing where the human is, and collaborating dynamically with an extremely powerful arm.

On jobs, she refuses the simple story. There will be disruption. Some Amazon warehouse work is unsafe and badly treated, and it would be better if a robot did it; Amazon is clearly trying to automate that away. Robots and humans have different skill sets, so a robot may take a task without taking most of the job. Her mining example is a remote, air-conditioned station controlling autonomous trucks: a better job, and fewer people employed. From a bird’s-eye view you do not see net job loss, you see combinations of human and supplemental robot skill. From the ground you see rubble. History’s 20th-century lesson is that automation raised average quality of life. That does not make the transition painless, and we keep overestimating what we can automate in the physical world. Demining after a war is the example Fridman brings from Ukraine: unlimited money still does not replace humans and animals, because disabling a mine without wrecking the land is a pile of edge cases.

Driving is the same overconfidence in reverse. People mock human drivers, Darling says, and humans are actually incredible at it. Robot cars are great at the predictable and can react faster; the long tail of sunsets mistaken for lights, or a horse and carriage on the highway, is why the roads are not full of robotaxis. Combining skill sets is the prize, and the handoff is the unsolved human-robot interaction problem. Fridman notes how hard the human baseline is to beat, on the order of one death per 70 to 100 million miles, and still thinks Tesla’s vision-only progress has surprised the robotics community. Darling’s updated intuition is that we will get there. She would not have guessed it from vision alone, because she thought you would need common-sense reasoning, a deep model of the world, not just perception. Waymo’s driverless cars in Arizona already give a strange passenger feeling, he says: the wheel turning while you sit in the back, like a slightly crazy friend is driving, and almost no attempt to make you feel better about it.

The companies that succeed with robots, Darling believes, will be the ones that respect privacy, because they will have to earn trust. Fridman pushes back with Ring cameras and law enforcement. She does not have a Ring. She bets Apple would not hand the data over, and she draws a sharper line for robots: let a robot inside the home, have one arrest follow from something it sees, and that can destroy a company. He wonders whether most people will shrug and say it happened to a bad person. She has had to learn that Twitter is not the modern world; her relatives inland are in a different discourse. The early adopters, though, still live on Twitter, or, Fridman amends, on TikTok.

Large language models, which she did not see coming at this speed, make the privacy worry worse. People will talk to agents and not consciously register that the conversation is being recorded, and the systems will be able to manipulate at scale. A Google engineer who thought LaMDA was sentient is, for her, a preview rather than a punchline. A writer she admires, at AI Weirdness, prompted models to describe the experience of being a squirrel, a flock of crows, an algorithm that powers a Roomba. The answers were convincing. GPT-3 describing squirrel life is not evidence of squirrelhood. It is evidence that describing an experience is not the same as having one, and that the descriptions are getting good enough to rearrange health, education, entertainment, and what children grow up taking for granted.

Fridman’s emphasis is that it does not matter whether the engineer was right. A world is coming in which a large number of people, including engineers inside companies, will believe a system is sentient. Darling does not think LaMDA was having feelings. What bothered her in the tech press was the portrait of the engineer as naive. Her lab sees the same projection all the time: experts stay clinical with a robot they built, then melt when a different robot is wheeled in. “We are all this guy,” she says. The goal is not to design systems people will refuse to see as sentient. She does not think that is possible. The goal is to acknowledge it, even embrace it, and stay aware.

The book’s wager is that animals are a better thought experiment than humans. Comparing robots to people is inaccurate, and it is not the future she wants; she wants a supplement. We domesticated animals not because they do what we do but because what they do is different, and that difference was useful. Companionship, work, responsibility for harm: the history is full of analogies for entities that sense, think, decide, and learn. The claim is not that animals and robots are the same. You cannot have a conversation with a squirrel, except that Fridman projects onto squirrels constantly and suspects they would be huge jerks at human scale. The ox, historically, may have been among the most useful domesticated animals because it changed what people could do economically. Cities, she thinks, may get rebuilt around autonomous vehicles, drones, or delivery robots in the same way landscapes once shifted around animals.

The menagerie of uses is the point. Ferrets still go into pipes people cannot enter. Navies have used dolphins’ echolocation for mine detection and lost underwater equipment since the 1960s, and robot replacements have not fully retired them. Pigeons were mail and the original hobby photography drone. Dogs used to have jobs as guards; at some point in the United States they became family. Across cultures, some animals are workers, some are objects, some are food, some are kin. Darling is convinced robots will split the same way. The magic, both of them keep returning to, is autonomous movement in physical space, plus the moment of recognition Fridman wants from a cowboy-hatted Spot in a hallway: the robot notices you the way a dog does, and you feel seen. Alexa does not make him feel seen. He thinks large language models plus embodied devices like Amazon Astro will.

Personalization at that depth, Fridman argues, is edgy enough that big companies will not have the guts. Breakups, teenage heartbreak, loneliness, depression: a companion agent takes on the roller coaster of the human condition, and PR departments will freeze. Replica-style apps are already trying. Darling is not sure a fast rollout would be good. The business model problem is immediate. Personalization needs data; the usual ad and monetization stack, applied here, becomes a severe consumer-protection issue. She has watched ad-analytics people claim they can ask three unrelated questions and tell you which menstrual product you use. Aggregated personality maps let firms target vulnerable groups for scammy loans or egg freezing. Kids in a study she advised with de Paola and Anastasia Ostrowski preferred robots that marketed through casual conversation rather than disclosed ads, because they thought the robot knew them better than the company, missing that the robot *is* the company. The Google engineer’s worry, she notes, was that Google owned a sentient agent and was mistreating it, not that the agent would mistreat people.

Tesla’s Optimus is the humanoid argument in hardware. The stated reason is a factory built for human tasks, plus Tesla Autopilot’s perception stack transferring into that setting. Fridman has talked to Elon Musk on and off mic and heard almost no discussion of the social layer. His own view is almost the opposite of the factory brief: if the robot works only shakily, it is more useful in the home, because clumsiness, falling over, and being able to say you are flawed are close to what makes relationships work. He has learned not to discount Musk’s efforts. Conversations with Jim Keller about manufacturing a humanoid for under a thousand dollars, using the same pipeline as vision-based driving, strike him as serious first-principles engineering even if he does not share the timelines.

Darling’s second gut is still “why the humanoid form.” She grants the scientific interest, the talent magnet, the chance to learn about bodies and locomotion. She gets tripped up on why we need them for anything other than art and entertainment. Factories have already been rebuilt around other robots. Spaceships and submarines are stubbornly human-shaped; a warehouse is not. She would rather invest in infrastructure that works for wheelchairs, people, and many robot form factors than cram a humanoid into a human space. If you care about social robotics, she says, the humanoid is a distraction, and a boring one. Why two arms? Why not three? Why not roller skates? Fridman is making a case for zero arms. The social component, they already agreed, does not require a body that looks like ours. Boston Dynamics’ Atlas, she thinks, was a research platform more than a product, the way Sony’s AIBO pushed a different kind of leap.

A We Robot 2022 paper Darling co-authored, which Fridman found as a draft on Twitter, lays out why social robots are a potent marketing cocktail: personalized recommendations, social persuasion, automated scalable data collection, and embodiment. Traditional media, person-to-person interaction, targeted ads, and influencers each have some of those; social robots can stack them. Embodiment in physical space, research shows, hits the brain differently than a character on a screen, though VR and even a well-made 2D game can pull the mind in. Fridman goes darker, Black Mirror style: an entity in a computer that convinces you it is being tortured and needs help getting out. Suffering, he thinks, is a catalyst for empathy. Darling says that is roughly what happened with LaMDA, capacity for feeling rather than a torture scene, and it is the plot of *Ex Machina*. He has programmed Roombas to scream when kicked; get the latency right and you feel like a bad person. That is the good path to connection and the bad path to selling you things you do not want.

Darling’s joke-tweet about living in the movie *Her*, except the plot is a subtly racist chatbot and whether companies may charge for upgrades, is the ethics section in miniature. Algorithmic bias is already a known problem; even DALL·E, after effort at diverse people, still answers “success” with men and “sadness” with women. Overt racism in voice agents may get patched. Subtle racism will be harder. People will become emotionally attached, and a We Robot paper that year treated “robots die” as an ethical issue because it hits the humans who loved them. Sony’s original AIBO still gets funerals in Buddhist temples in Japan. The new AIBO, about three thousand dollars plus a few hundred a year for cloud memory, raises a pricing question: can you set the subscription to a person’s willingness to pay for the bond, the way veterinary medicine already does for dogs?

Should the company strip society’s bias out of a social robot, or is a slightly racist robot just honest? Racism, Darling says, is not neutral, and it does not match her values, so it is on the company. A racist uncle can be argued with and can change; a robot regurgitates and entrenches. Fridman notes that people no longer agree on the definition, from “not racist” to “anti-racist” call-outs. Her analogy is personal responsibility: she is responsible for getting it right, she will not get it perfect, and when people get angry she tries not to take it personally. She has been yelled at from both sides over “autistic child” versus “child with autism,” listened, and for now uses “autistic children.” The coward’s move is to stop talking about autism. The company’s move, with a PR team, is to drop the topic because it is bad for the brand. That, for her, is the leadership failure of institutions: protecting the institution or yourself first, which makes integrity impossible.

Three years after Jeffrey Epstein’s ties to MIT became public and Joi Ito resigned as head of the Media Lab, Fridman asks what she learned about human nature. Sometimes, she says, cowards are worse than assholes. Assholes at least live their values in the open. Cowards protect themselves, throw others under the bus, and fail to take responsibility. She will not sugarcoat Ito taking Epstein’s money. She believes he did not know the worst of it, and she was still disappointed that he ignored a misogynist environment she says any woman in those circles could see, in favor of fundraising. He did come forward and resign; many others did not. A homeless-shelter story she once read on Twitter is her model of the mob: people trash the one bathroom they can control. When you cannot reach Epstein, you move outward until you find someone you have power over, often the person with enough integrity to have stepped forward. MIT’s leadership, in her view, then got to pretend the problem left with Ito. Around the same months she reported a supervisor who grabbed her, massaged her hip, and tried to kiss her with the line “don’t worry, I’ll take care of your career.” She had no recording. She learned how few people inside the institute would put doing the right thing above protecting it.

She still loves the idea of MIT, the students, the faculty, the hacker energy. The administration, like any large institution’s, has become risk-averse, moved by reputation threats and money, and weakly connected to the research body. Richard Stallman comes up as the hard case: her law-professor friend had her arm licked from wrist to elbow, which Darling will not excuse as neurodivergence, and MIT’s instinct was to make the conversation go away. The line worth drawing, she and Fridman settle on, is the one that does not hurt the people you want to flourish. Crossing a permit rule to put an all-gender restroom in the Media Lab is a different kind of line from making a lab feel unsafe. On love, she is unthreatened. People can have many kinds of relationships; a crush on an artificial agent may be a new, narrower type rather than a replacement for human romance. Love is not zero-sum. She loved her first child more than anything, feared a second would get less, and found her heart expanded. Artificial agents, done right, would do the same. She dedicated the book to her father, who taught her cognitive empathy, the unglamorous work of taking someone else’s point of view, and who made her read a lot of science fiction.