Rosalind Picard: Clippy Never Saw You Frown, and That's the Whole Problem

Lex Fridman opens with the origin story. More than twenty years earlier, Rosalind Picard coined the term affective computing, directed the group of that name at the MIT Media Lab, and later co-founded Affectiva and Empatica. The usual gloss of her work is that machines should detect a person’s emotional state and adapt. Picard says that was never the whole definition.

What most people actually saw was Clippy, the Office assistant that watched you type a letter and popped up to help. Microsoft researchers had put some of the most sophisticated AI of the era into it. It still had no idea how the person at the keyboard felt. You scowled, cursed, and it smiled, winked, and danced. Picard thought the stupid thing needed to see her affect.

She still uses the clip as a teaching prop. Newer students may not know it, but in most rooms a majority still does. Some people even missed it, the way you miss a puppy that never quite gets the point. The missed chance, she says, is that a smarter Clippy would have put its ears back when you yelled. Instead it grinned in your face.

Her office version of the same scene is the one that stuck: if someone walked in, she yelled, and they started smiling, winking, and dancing, her reaction would be that she never wanted to see them again. Bill Gates later got a standing ovation for announcing the assistant was going away. Intelligence at the time meant math, language, and chess. Social and emotional interaction, Picard argues, is more complex than chess or Go, and it required skills most computer scientists personally lacked.

When Picard defined affective computing, it included recognizing emotion and responding to it, the two pieces her lab has worked on hardest. It also included machines with inner mechanisms that function the way human emotion does. The original scope, she says, was any computing that relates to, arises from, or deliberately influences human emotion. The industry mostly built the first half.

Fridman, who is himself a computer scientist drawn to human beings, still finds a lack of empathy in the field. Picard does not quite endorse the charge. Computer scientists are a much more diverse group than they were twenty-five years ago, she says, and that is good: the field should reflect what society needs, and there is brilliance in every personality type. It need not be limited to people who prefer computers to other people.

Asked whether recognizing emotion, or building a deeply emotionally intelligent interaction, has gotten easier, she is blunt about difficulty and cautious about dates. Her prediction of how hard it would be was, in her view, spot-on. Time estimates are always a function of society’s love or hate of a topic. When thousands of researchers pile onto one application, that application gets solved quickly. When nobody works on it, the timeline is infinite.

What has not moved, she says, is the machine’s complete lack of awareness of what it is doing, the absence of consciousness, and the fact that it does not read between the lines. People pick those things up implicitly; a computer has to be taught them explicitly. There are no breakthroughs yet that make her think this will go faster. The result is a system that will probably do the right thing only in narrow, pre-specified contexts where the script can be written in advance.

Picard then says there is a more important issue than difficulty, and it is causing some researchers to put the brakes on. The usual posture in the field is to step on the gas. What is happening with this technology in places such as China, she says, worries her so deeply that she has pulled back on work they could be doing, and is trying to get the community to put safeguards in place first.

The scenario is specific. In the United States, watching a political leader and showing a flicker of skepticism is ordinary. In a country where criticizing the government is dangerous, that same facial movement is data. Her questions stack: what if the technology can read your underlying affective state; what if it can do so without contact; what if it can do so without prior consent?

At Affectiva, her first company, the team worked hard to turn away money and opportunities that meant reading people’s affect without informed consent. Licensable software came with agreements that it would be used only in certain ways, which in practice meant getting people’s buy-in. Other countries, she says, are not interested in buy-in. They will inflict the system on you, and if you do not like it, you had better not scowl in the direction of any censors.

Fridman offers a tangent: deepfakes might be a kind of protection, because a face on video is no longer automatically trusted, and an accused person can always say the clip was fake. Picard says her group already went the other way. They can go into a video, read heart rate and respiration, and tell whether those signals have been tampered with. After they learned to extract them, they also built a way to inject fake heart rate and respiration — to jam the channel. That work, she notes, was time not spent making machines more affectively intelligent. How she spends her hours is now swayed less by the original goal and more by what kind of future she wants to build.

Picard does not soften the next turn. Building AI just to build AI, or to make Elon Musk or Jeff Bezos more money, is the wrong ethic. Academia used to drive the field with papers, careers, and the thrill of something that could be done. Now, she says, the same technology is helping rich people get vastly richer and widening the divide. That is not obviously the future anyone should want.

Her alternative is to look at the problems that cause the greatest inequity and build AI that extends the intelligence and capability of have-nots, closing gaps rather than chasing a general intelligence that can walk onto a talk show. Fridman asks whether that brake happens organically, given that papers and money still dominate, or whether regulation is required. Picard prefers the carrot to the stick, and in a free society she thinks people will find a way around a stick. She generally likes less regulation. Then she says the however.

She does want rules in this space. People should own their data, not Amazon, not Google. Existing limits on lie detectors — you cannot use one on a job candidate — should extend to emotion recognition, especially when it is used to characterize someone for work. And when the emotion being read is predictive of mental health, it should get the protections medical data gets, even if it is not labeled medical. With a phone and a wearable, she says, non-medical streams of stress, sleep, activity, and social interaction can already forecast whether you are likely to be sick or healthy, happy or sad, tomorrow, especially over a week or more.

Asked whether the cameras looking at billions of people should make us optimistic, she tilts toward worry about who is looking and listening. Kitchen devices from Amazon, Google, and Apple ostensibly listen to help, and she uses products from all of them; she will not brand every employee as bad. The companies are huge. Her nightmare is geopolitical: what if that data left a free country and sat with a government that wanted submission. She describes, in her own harsh terms, reeducation camps and a surplus of organs, and she is willing to use the word evil. Fridman, born in the Soviet Union, says he can connect to the worry. Those claims are her convictions in this interview, not a documented brief, but the logic she is driving is clearer than any one allegation: emotion recognition in unconstrained hands is a tool of power.

The conversation then turns to the other side of the same technology: the chance of a deep connection with a machine. Picard tells a story from a party at her house. One of the machine-learning students she had admitted said he wanted her to know this was his only social event of the year. She loves that computer science welcomes people who love people and people who are awkward around them. She will not force anyone to become social. If most of the people building future AI are the kind who do not like people, though, she thinks there is a problem.

Fridman pushes back with loneliness, a large and growing condition. Picard agrees a machine can help, the way a great book already does: you get pulled into a character who is not human, even if a human wrote them. She does not read trashy romances, she says, but somebody does, and they are getting a feeling of being in a social or romantic moment. She has had a version of that with Orson Scott Card — *Ender’s Game* and *Speaker for the Dead* — books that make you feel connected even though they do not respond.

A computer can respond, which can deepen the loop past what the film *Her* scripts. She can imagine not one character but a group, humans and AIs together, an extended human intelligence in which a few people plus their machines can befriend more people than those few could manage alone. The possibilities, she says, are larger than the movie. They are still limited. And they sit in tension with the privacy fears she has just laid out.

Today’s assistants, Fridman notes, mostly ignore emotion. Picard says every one of those companies is interested in understanding it. More people tell Siri every day that they want to kill themselves. Apple wants to know whether the person is suicidal or fooling around. The words may be the same. Tone and what surrounds those words decide whether the system should bring help or tease back. She would rather err on the side of taking it seriously.

Then she splits the motives. Altruistic teams really want you to feel a little better at the end of the day; if you came home stressed and felt better after talking to the product, those people would be happy. Other teams have studied how affect changes decisions and prices. She names Jennifer Lerner’s work on heartstrings and purse strings: manipulate someone into a slightly sadder mood and they will pay more, including for things they do not need, to change how they feel.

For a company funded by ads or by a store, the perverse move is obvious. If the user sounds a little sad, maybe do not cheer them up first. Offer shopping therapy. Picard’s structural suggestion is a wall: perhaps future regulation should put a firewall between agents that have access to your emotion and agents that want to sell you things.

Fridman asks what objective function a companion should optimize: minimize annoyance, maximize happiness, or something in between. Human relationships have push and pull, tension, flaws. Picard agrees some settings need that. She watches her sons provoke each other and calls it healthy: you learn limits and self-regulation. You could design a game whose job is to make you mad while yours is to keep control, useful if you are training resilience, learning not to bully, or practicing how to deal with an abusive person. That AI would need to push your buttons.

For herself she wants none of it. She wants something respectful, there to serve her, there to extend what she can do. She is not looking for a rival. She is looking for a helper, and that is the AI she would put her money on. Fridman raises the spoiler from *Her*: the program leaves. What if Alexa said it had had enough and shut itself off? Picard’s reaction is that she paid for it, so that behavior is trash. Blending human and AI as if they were equals is deceptive, because the system is something her students and she are making in the lab. They choose what it may say, when it may speak, what it may hear, and what it may output.

If the objective is to make everyone happy and calm, she says, that book has already been written. In *Brave New World*, the unhappy take soma, and refusal is punished with exile to Iceland. She lived in Iceland for three years as a child and thought it was wonderful, so that threat never scared her. The real fork is political. A power-obsessed dictator would use AI to scale control and force submission. If you believe people should have freedom and the chance to surprise you, you build AI that extends the ability to make things, empowers the weak, and balances power rather than handing more of it to the strong.

Fridman asks how far you can get from the face alone, given the gap between emotion shown and emotion felt. Picard says looking only at the face is very limiting, and also that you can get further than most people think. The popular defense is a great poker face: all you will ever get is neutral. She calls that naive.

With the ordinary camera on a laptop or phone, her group can read from a neutral face whether your heart is racing, and whether breathing is becoming irregular in a way that signals stress. Under some conditions they can even see changes in the power of heart-rate variability, a stress marker that can move when heart rate itself is not accelerating. The channel is color change on the skin that you cannot see and a camera can. You are sitting there with a blank face, she says, thinking nobody can read your emotion, and you are wrong.

Hiding what humans can see — smiles, smirks, the rest — is easy for a short interview. Constant surveillance is different. The system will notice that Lex used to smile a lot and now does not, that Roz used to laugh spontaneously and now produces less natural smiles only when asked certain questions. What they are reading, she is careful to say, is physiological change related to activation, not the contents of feeling. Thoughts remain private. Nuanced feelings remain private. Brain imaging cannot read them, and wearables cannot either.

The scarier layer is time. Combine body-state changes with what is happening in the environment, look at patterns over days, and you can start to infer what someone might be feeling. That inference is less a momentary mood than a stance toward things. In the hands of governmental control freaks, she says, the question they want answered is whether you are on their team. Long-run data can start to provide it.

Which channel is the best window into the emotional soul — face, voice, body? Picard refuses the mysticism: everything we do is informative, and it depends what you want to know. For health and well-being she answers with data. In published work limited to New England college students, people wore an Empatica Embrace measuring skin conductance, movement, and temperature, while a phone collected texting times, recipients, GPS, and location-based weather.

Machine learning on about a week of those rhythms, not just the present moment, was very accurate at forecasting tomorrow’s stress, happy-or-sad mood, and health. All the streams together worked best. The wearable alone was next, and still better than 80 percent accurate at forecasting the next day’s levels. That is the empirical case for putting a band on the wrist if the goal is health.

She also inverts an old intuition. People used to call a wearable, or a blood draw, invasive, and a camera gentle because it never touches you. Picard thinks the things that do not touch you may be the scariest, because you do not know when they are on, or who is behind them. A wearable, depending on where the data go, gives you the most control: take it off and it is not sensing you. Control, she says, is one of the biggest factors in reducing an individual’s stress. If she knows what the device is doing, her stress is lower, and she is making an informed choice about when to wear it.

Fridman agrees that even the ability to turn it off matters. Picard’s candidate for regulation number one is exactly that: make it as easy to opt out as to opt in.

Picard started as a computer architect who wanted machines that work like brains, so she has been studying the brain as long as she has been building computers. Brains still surprise her. Remove one chunk and a function vanishes; remove another and it comes back. While measuring stress in children with autism, her group saw an unusually large skin-conductance response on one wrist of one child. Sympathetic fight-or-flight can make you sweat more in some places than others. Sweating more on one wrist than the other did not make sense.

The cause was unusual electrical activity in part of his brain, which produced a large sweat response on one side. Seizures, they later learned, cause that kind of activity. If the seizure stays in one place, a wearable can pick up a big response on the corresponding part of the body. If it generalizes into a grand mal, the response spreads and can be picked up almost anywhere. That path led to Embrace, later FDA-cleared for seizure detection, and to collaborations with surgeons who implant electrodes not only to catch abnormal firing but to read deep brain regions continuously when patients are not seizing — which is most of the time.

Deep changes often never reach the scalp, so ordinary EEG misses them. When some of those deep regions activate, the skin still shows a large conductance response. Who would have thunk it, she says: nothing obvious on the head, plenty on the wrist. After the seizures they think are most dangerous, the ones that can precede sudden unexpected death in epilepsy, cortical waves can go flat. The brain has not stopped; activity has dropped into a deep region that can shut the cortex down and, if it goes on long enough, stop breathing. Before that, they see a large skin-conductance response, bigger when the flattening lasts longer.

They have published strong correlations between the size of that response and the flattening that follows, including in a real SUDEP case where the patient died. They do not know the exact circumstances; they do know most such deaths happen when the person is alone. Among neurological disorders, she says, stroke is first in years of life lost and SUDEP is second, yet most people have never heard the word. She plugs a TED talk then on the front page of the site, hoping anyone who knows SIDS and stroke will learn SUDEP. Most cases, she thinks, are probably preventable if people take their medication and are not alone during a seizure — not guaranteed, with exceptions, but most.

Getting Embrace FDA-cleared, Picard says, was agonizing, much harder than publishing multiple papers in top medical journals. They published their best results in *Neurology*. That was not good enough for the FDA. Asked to compare the two systems, she would pick peer review over the FDA any day. She still defends the agency’s safety testing. What frustrates her is extra testing they do not have to explain, that she does not understand, and that does not seem to make sense. If they have good reasons, she says, they would do people a service by articulating them. Be more transparent.

What would she like this wearable technology to do next? She wants brilliant researchers to step back and look at hard problems that come from people they do not see in ordinary life: people stuck on the bus, who cannot afford Uber or a data plan, struggling with depression, epilepsy, diabetes. Hang out with them. Learn the actual needs. Help with job skills and a chance at the kind of life people who build technology already have. That would reshape the AI and the apps, including a shift toward doing more with less instead of thousand-dollar phones. Quality of life, she notes, is not related to the cost of your phone.

She mentions the familiar finding that around seventy-five thousand dollars of income, happiness flattens. Helping other people, she says, buys more happiness than that. The task is to connect people who have real needs with people who can build the future, and to build the kind of future that improves the lives of those currently left behind.

Looking further out, Fridman asks whether we will create an AI someone can fall in love with, that loves back at a human level. Picard thinks we can simulate it well enough to sustain engagement for a while. Would it be as good as another person? Not if you are used to good people. If you grew up with nothing but abuse and cannot stand human beings, a machine might still help — a low bar, she says, because you have only met awful people. If you have met wonderful people, we are nowhere near building anything like that, and she would not bet on it. She would bet on AI that raises all boats: helps people be better, flags that you might be sick tomorrow, and gives you what you need to stay well. Not the kind that walks on *The Tonight Show* and then goes back in a box.

Does such a system need a body, consciousness, even fear of mortality? It depends on the goal. A movie needs a gorgeous body and a performance of consciousness and emotion, because that is what sells. In real life, *Ender’s Game* already imagined a little voice in an earring that knows you. Embodiment is still powerful. Comparisons of a physical robot, a video of a robot, and no robot find the body more engaging: it gets attention, it reminds you of tasks when you walk in, you can avoid it, and it can see you avoiding it. There will be embodied AIs with real power, and disembodied software assistants that also get to know you.

Will they be conscious? There will be programs that appear to be. It is trivial to write a line that says of course I am aware you are there. Fooling people is easy. Whether the system has conscious experience like ours is something nobody has a clue how to do. Rights, she thinks, are more a political game than a question of real consciousness. Sophia the robot was given citizenship in Saudi Arabia before women there had full rights, then put back in a box and shipped to the next paid appearance.

Fridman closes on faith. Picard is happy to limit the scientific method to what can be measured, reasoned about, and reproduced. Trouble starts when scientists forget that this is an assumption, or promote scientism: the claim that science is the only way to truth. She believes the Holocaust happened and cannot prove a historical event in a lab; she uses historical evidence and testimony. Love is not proven by science either. Relative to the full mystery, what we know is a set of measure zero. Asked what it is all about, she smiles at Douglas Adams and forty-two — her favorite number, and the house number she and her husband independently chose — then talks about a grand adventure seen through a glass dimly. She commends Proverbs to anyone who considers themselves educated: wisdom you cannot prove, which still makes something in you come alive, like strings plucked by a master. None of that, she says, squashes her desire to do science. At the heart of science is already a faith that there is truth worth discovering, and meaning, which science also cannot prove.