Kate Darling on Why We Name the Roomba and Mourn the Bomb Robot

Kate Darling, a researcher at the MIT Media Lab, keeps three Pleo dinosaur robots at home named Yoki, Peter, and Mr. Spagetti. She studies the emotional connection between people and lifelike machines. Fridman starts with a Harvard reading group she once ran on robot ethics. If she taught it today, she says, the syllabus would look very different. The phrase still sounds like science fiction, but the live issues are concrete: responsibility for harm, automated weapons, privacy and data security, automation and labor markets. Personally she is drawn to the social layer — one-on-one relationships with robots.

At the societal layer, Fridman brings up Andrew Yang, then running for president on a warning that automation will take jobs, plus a universal basic income of a thousand dollars a month. Darling lived in Switzerland for twenty years, where UBI was already a topic, and not because of robots. What interests her is watching Silicon Valley latch onto a concept that came from a left-wing, socialist corner of Europe and rebrand it as an answer to machines.

On labor itself she is blunt. Those conversations often overestimate where robotic technology is, and they commit a fallacy she will spend the hour dismantling: comparing robots to humans as a one-to-one job replacement. Bill Gates once floated taxing robots for taking people's work. Darling assumes it was taken out of context — he is a smart guy — but the line still sounds like substitution rather than a supplemental tool. The technology will shake up the landscape. She does not see robots taking all the jobs in the next twenty years. That is not how it is going to work.

Fridman asks the darker question: will people abuse robots, especially once they are treated as servants? Darling does worry. People already treat robots like living things even while knowing they are machines. Whether that counts as abuse is another matter. You are not harming an inner mind; as far as we know the robot has no feelings. It depends how you define feelings and consciousness, she adds, but people overestimate the tech. Current robots are not even as smart as insects. She is not worried about abuse in that sense.

The live question is what violent behavior toward these objects means for us. Does verbal or physical roughness desensitize people? Violent video games are the usual analogy, and after decades the research is inconclusive. Society has mostly decided adults can compartmentalize a screen — shooting characters or running people over in a game does not mean you will do it in the street — while remaining more cautious with children, evidence or not.

Robots may not inherit that truce. A lot of research, she says, shows we respond differently to something in physical space than to something on a screen: more viscerally, more like a physical actor. It is possible this is still just video-game violence in 3D. It is also possible it is not. There is no evidence yet, so the question has to be asked again. Maybe, she later allows, some people need an outlet — a robot to scream at instead of a person. A friend of hers put the other hypothesis more sharply: it might just train their cruelty muscles.

Darling is writing a book on the history of animal domestication — work, weaponry, companionship — because she thinks that history is a better predictor of how we will treat robots than any human-to-human analogy. Throughout history we have treated most animals as tools and products, and some of them differently. People are already treating robots in similar ways. Fridman wonders whether, a hundred years from now, we will look back at what we do to animals the way we look back at the Holocaust. She hopes so. She is not convinced we will. She does wonder what her grandkids will find abhorrent that her generation shrugged at.

Fridman, who anthropomorphizes robots and sees no fundamental philosophical difference once capabilities match, can already imagine a civil rights movement for machines. Darling says she started there too: no deep difference between technology and human consciousness, so equal rights once the levels match. Then she noticed that is a far-future question. We might need to ask the "robot rights" question sooner, while the machines are still, quote, really dumb — because of how we perceive them.

The animal-rights lesson is that we have always protected what we relate to most. No one gave a shit about the whales until someone recorded them singing. Then they were a beautiful creature, and the 1970s Save the Whales movement began. People want to believe we care about consistent biological criteria. Historically that is not how alliances formed. You can still shoot a Roomba; it is not murder. You can also shoot your dog, she notes, but people will not be happy about it. We forbid torturing a dog in most places and still treat it as property in others. The lines are arbitrary. Calling humans unique, in a lot of philosophical work, is just speciesism.

Fridman asks whether most people are good. He has studied the Holocaust; he is Jewish; he believes most of us have evil in us, and what bothers him about robot abuse is that it reveals that evil. Darling does not share the bother so much as the opportunity. Some of her research suggests behavior toward even simple, unfeeling robots says something about a person's general empathy. *Westworld* is not so far off: the bad characters shoot and rape the hosts, the good ones will not, even without assuming consciousness. Robots become a place to practice the better angels.

Some people ask why we would practice empathy on robots instead of on fellow humans or on animals that are actually alive. She disagrees. Empathy is not a zero-sum game. It is a muscle you can train, and we should. Fridman mentions parents who make kids say please to Alexa. He is himself polite to speech systems, including the car, but he is not sure that lesson is the right one. Darling plays devil's advocate: if you have abusive tendencies, a robot you can scream at beats a person. Then she knocks her own argument down. We do not know whether that is an outlet or cruelty practice.

Sex robots fatigue Darling. Tell someone you work in human-robot interaction and it is the first thing they want to talk about — a titillating news hook that eats the rest of the field. She still thinks the topic makes some interesting questions salient. Fridman's bet is that men who buy female sex robots will find the interaction, not the sex act, is the fulfilling part, like the movie trope of paying a prostitute and talking all night. People joke about sex. Many of them are lonely and looking for connection, and it would be a shame if that connection only came in through the sex industry's side door.

Darling wants the sex industry destigmatized as well. Some sex workers specialize in disabled people who do not have the same chances to explore their sexuality; that need is real. The replacement panic — if the sex is really good, people will not want their partners — misses the other case. Robots can fill a hole where there is nothing. Fridman pushes further: asking family and friends to explore the full complexity of who you are is already too much; they do not have the patience. Darling keeps the frame plural. Humans are capable of many kinds of relationship — family, friends, pets, robots — each valuable in a different way. Nobody has to stand in for everyone else.

Most of Fridman's work is autonomous vehicles, so the trolley problem arrives on schedule. Darling thinks people have forgotten its origin. It was meant to show that there is no right answer, and that an instinctive moral flash is not what you should follow if you care about systematic rules for everyone. MIT's Moral Machine — a site, built by friends of hers, that pits two pedestrians against a child, a homeless person against a jaywalker — produces interesting, valuable data. It should not become the rulebook for cars. Crowdsourcing first instincts is exactly what the thought experiment was trying to undermine.

The practical version is ugly. If your life outranks a stranger's, the stack is more likely to swerve. Today's automated emergency braking only brakes; it never swerves, because leaving the lane can harm others even if it saves you. Who buys a car programmed to kill the owner instead of someone else? In reality, Darling thinks manufacturers will dodge the decision and the liability, and they will protect the driver. The harvest is the insight: forcing ethics into code reveals that our rules are much less programmable than we thought. Legislators already do this work — suss out what a society agrees on, write it down, adapt as culture moves. The Moral Machine clicks are not that. Asking "would you run over the homeless person in this scene" is not the same as asking "do you want cars to always run over the homeless." Put the same people in virtual reality and the answer would change anyway.

Anthropomorphism, as Darling defines it, is the tendency to project human-like traits, behaviors, and qualities onto non-humans. We do it to animals, teddy bears, faces in car headlights — and to robots, extremely. Design can enhance it. It does not take much. Over 85 percent of Roombas have a name, a share no ordinary vacuum cleaner matches. People feel bad when the disk gets stuck. They send it for repair and want the same one back. None of that was designed in.

Amazon, she has learned, has a whole team on Alexa's personality, and the personality differs by country — German Alexa is not American Alexa. Still, voice assistants are a hard place to harness the illusion. The interface is primitive. To get people to suspend disbelief, less is sometimes more: let them project, and do not disappoint. Alexa and Siri talk too much and are not good enough. The baby seal used with dementia patients is simpler. It does not try to talk. It moves, makes sounds, responds to touch. That is a very effective way to be treated as alive.

The bad cases are when a tool is supposed to stay a tool. In the early 2000s, iRobot's PackBot went to Iraq and Afghanistan with bomb-disposal units. Soldiers named the robots, gave them funerals with gun salutes, and were traumatized when one broke. That is fine until a soldier risks his life to save a machine built so a person would not have to. Companies can also leverage attachment to sell products, services, or a cloud subscription for a social robot that is not in the customer's interest. Connection is not the scandal. Monetizing it against the user is.

Asked for a favorite real robot, Darling names two. The first is Pleo, the 2007 baby-dinosaur toy, about the size of a small cat, no longer sold. Motors gave it lifelike movement; it had touch sensors and an infrared camera. What struck her was how well it mimicked pain. Hold it up by the tail and a tilt sensor tells it which way it faces; it squirms and cries out. Hit it too hard and it cries. She was impressed by how much she bonded with it, how much she wanted to believe it had an inner life.

The second is Boston Dynamics. She had been the person who watched the videos and assumed they were product shoots. Visiting in person, she was very impressed. Fridman calls seeing Spot Mini a transformational moment: maybe a psychology experiment, but he anthropomorphized the crap out of it and it immediately felt like his best friend. The arm on Spot Mini looks like a head. The company says they did not mean it that way. It looks exactly like that anyway. The platform is not supposed to be intelligent — a movement base on which customers build higher intelligence, actually a really dumb robot. Even dumb robots get a visceral response.

Sophia she files as an art experiment, a puppet, much less impressive than Boston Dynamics. Science fiction and pop culture, she thinks, make people overestimate how far along we are. Fridman was surprised how little people cared — they treated it as normal, when if they understood the difficulty they should have been more astonished. Her favorite fictional robot is WALL-E. Animators since the 1940s have known how to make characters that do not look real but look better than real. Bambi was supposed to be lifelike until someone noticed that huge eyes, a small nose, and big cheeks — a baby face — made people like the deer more than accuracy did.

Could Alexa in the home go "better than real"? Darling would bet those companies are working on it behind the scenes. Fridman disagrees: Amazon wants you addicted to the service, not the cylinder, because a device-level bond creates more room for frustration than for love. Get it wrong and you get Microsoft's Clippy. Fridman, who has an army of Clippy defenders, calls that clumsy paperclip the greatest assistant ever built because it was a real attempt at personality, however premature. Darling agrees to be slightly less hard on Clippy.

Anki and Jibo, two celebrated social-robotics companies, had recently shut down. Darling does not think the difficulty is forever. Right now it takes enormous work to barely meet minimal expectations that science fiction inflated — people want Rosie from *The Jetsons*. The design on those products was beautiful. People wanted more functionality. There has been no killer application useful enough for mass adoption. Owners who did get Jibo became very attached, which is the Palm Pilot pattern: most people cannot see why they would need it until they have it, or until someone else makes it a little better. Expectations keep shifting with whichever fiction is popular. Need for connection, Fridman thinks, is already high, and we are not aware of it. We have gotten used to never really being close to anyone. We are deeply lonely, all of us, including people in deep relationships. Darling agrees.

Can you fall in love with a robot? Totally, she says. A long-term committed monogamous relationship? There are many kinds; "monogamous" smuggles in a Facebook relationship status that may not apply. She would rather compare robots to pets: they fulfill loneliness in a different, supplemental way. Some people will want to marry the machine. She does not think that is the main use case. Closing the gap between husband and pet someday is possible. Why would we? Recreating what we already have is boring; we could create something different. Even *Her* kept the difference — the AI went somewhere the human could not follow. In five or ten years, at best, robot relationships will be closer to pets than to people.

Human-level conversational intelligence, Darling thinks, is not required for anthropomorphism — intelligence sometimes gets in the way. Passing the Turing Test may be easier than solving pure natural language processing, because you can cheat. It is all expectation management. If you tell people they are talking to a 13-year-old boy from Ukraine, they will not expect perfect English, perfect concepts, or even the same conversational wavelength. That is the ELIZA pattern in a new costume.

A body helps, because we are physical creatures and we grant physical things more social agency. A body can also ruin it. She has a Hasbro robot cat that is disturbing to watch: she keeps expecting it to move like a real cat, and it does not, because it is a hundred-dollar gadget. Disappointed expectation kills the illusion. The trick is the same as the Ukrainian teenager: pick an animal people are not intimately familiar with. A baby dinosaur — nobody has actually held one — lets you get away with much more. As she put it in a TED talk, nobody knows what a dinosaur looks like.

Faking a fear of death, a self, an identity, is another design tool, neither inherently good nor bad. Pleo's distress is a powerful way to get a certain kind of engagement. A research partner, Nandy, built himself a robot with a lifespan that would stop working after a set time, to see whether he would treat it differently. Tamagotchi was extremely primitive, and people still responded to the idea of mortality. Small tricks make people do a lot. Whether that is good depends on the aim. A deeper connection for the user's own benefit sounds great. The same tricks can serve advertising, data collection, or a government extracting information.

How do you stop companies from crossing a line on emotional and behavioral manipulation? Darling says it is difficult because the products now need data to work, and even consumers have little incentive to curb that. The harms are not tangible. They trickle down at a societal level, and then suddenly you are living in *1984* — a book she found very prescient. She has an Echo. She tells Alexa all sorts of things. Alexa knows the household diaper brand, so reordering is easy. She has no personal reason to ask a lawmaker to stop it. When that same data is used against low-income people to target scam loans or education programs, the societal effect is severe.

Fridman adds the indirect version: nobody has to sit there petting an evil cat. Set the objective to maximize engagement and whatever follows is only indirectly your fault. Companies act in companies' interests; when those line up with people's, fine; a completely free market fails on information asymmetry. He also does not think Facebook and Google always know what is good for society, even when doing good would help the brand. Darling says something she admits is controversial in her circles: she does not think every company that hires ethicists and stands up an interdisciplinary board is merely whitewashing. A lot of them care about the right answer, do not know what it is, and are looking for help. Vilifying the cat-stroking villain is too easy. Plenty of people inside firms are well-meaning. What we absolutely need is more interdisciplinarity — inside companies and in policymaking — because technological progress has hurtled ahead and you need people who understand the tech talking to people who understand the social effects. There is no other solution.

Europe has been trying to legislate algorithmic transparency: if a system makes a decision that affects someone's life, you should be able to see how. Darling does not work on that herself, but she thinks the balance is real — companies need some competitive advantage or you stifle innovation, and for some current uses people still need to understand how the systems work. Intellectual property in software and AI, she says, is simply broken. Patent law is a wrench. It works well for pharmaceuticals. Apply it elsewhere and you are hitting the thing with the wrench and hoping.

Code is automatically copyrighted, but a slightly different implementation walks around it. Software patents are possible and, in her Ph.D. experience at a university, insane: so many lawyers and meetings that it felt like hundreds of thousands of dollars. The garage inventor is a thing of the past. By the time a patent issues, often five years later, the code is obsolete. The MIT Media Lab's default is open source, for a practical reason. Students ask how to protect their code. The staff say that is not the problem. The problem is getting anyone to use the project at all. There is so much stuff out there you do not even know if you will get traction. Open source can spread the work and still keep attribution.

Raising a child, she says, has made it more apparent that current systems are not like human intelligence; her son has learned in such a different way that the comparison is not interesting. What is interesting is the world those systems will shape for him. She is even more concerned about the societal effects of massive data collection. Looking five or ten years out, what she wants is a home social robot that is not just a cylinder. She loved aspects of Jibo. She hopes a company cracks it. The conversation ends where the business case stalled: expectation, function, and a species that cannot help attaching to machines — which is both the product opportunity and the ethical risk.