People Are Easy to Predict — and They Hate Being Told So

Long before smart speakers, Charles Isbell wired his house and recorded every infrared blast from the remotes — lights, television, the small ritual of sitting down to watch something. Georgia Tech already had an Aware Home that logged this kind of traffic without cameras. He wanted a narrow answer: how much data do you need before a model can guess what a person will do next with a TV or a lamp. The surprising part, he says, is that capturing it is easy once you know what you are looking for.

It has to be the right two days, a weekend rather than a random weekday. With that slice, a student could predict the next button on the remote at 93 percent accuracy using counting statistics — no hidden Markov model, nothing deep. Widen the question to the little cluster of buttons you might press, and the number climbs to about 99 percent. The keypad digits, for example, all behave the same way with respect to a person, so they fall into one pile, exactly as you would hope.

The deeper move in his work is to represent people and objects as distributions over actions taken and actions suffered, then cluster those distributions. Individuals differ, but any given individual is irritatingly regular. After years of staring at this, Isbell has two conclusions: people are easily predictable, and people hate when you tell them they are easily predictable. They still are.

Lex Fridman pushes back in the obvious philosophical direction. If ninety percent of behavior clusters, maybe the human part is the outlier — the moment someone breaks the routine, and the way each of us breaks it. Isbell grants the poetry and disputes the premise. People who look extremely different from the crowd are not zero percent of the data. They are closer to ten percent, and when you cluster *them*, they look like one another too. Even the people who fall out of line have a line.

He restates the question. What do you do when the situation is one you have never seen, when you are forced to act, and when the right answer — if you even know it — is still going to hurt? That, at least at a philosophical level, is what makes an individual, and what makes people in general. It is not the easy problem and not even the merely difficult one. It is the outcome you know will be highly suboptimal for you. Hard problems are the ones that define us.

Fridman asks whether interactive AI could pull people out of social-media silos — Democrat and Republican as two silly bins among many — and help them feel someone else's life. Isbell does not think the geometry is the hard part. If you can put a partial order or a clustering on ideas or behavior, the way from A to B is not a leap. You find the nearest edge and walk it slowly. Machines are good at that, once a designer has named the edges and given them meaning.

The network itself is the trap. He retells a story: two groups from the same background split to solve a problem, then fail to merge, because a few hours of work invented two languages. One group, looking at a room, can see a clock and names the room by the clock. The other group cannot see it and uses some other landmark. They return talking about the same room without knowing it. Worse, neither side knows what the other can see. If you manufacture silos, the fight is not mainly that the ideologies disagree. It is that the same words mean radically different things, and people talk straight past one another until they agree on a few axioms.

Fridman folds in empathy, including the online habit of treating the word *love* as cringe. Isbell splits empathy from sympathy. Sympathy is feeling for someone; empathy is understanding where they come from. For most people the two travel together, but they can be pulled apart. You can understand a person completely and still refuse to support what they do. If understanding required agreement, or if feeling for someone required making them like you, we would be lost, because we are not the same person. His fanciful engineering move is to treat each life as a distribution over experiences and actions, take the cosine, and introduce people at the overlap. Almost nobody is a zero. Almost nobody is a one. The real bottleneck, he suspects, is getting people to ask how they are more alike than different. If they will ask, they may not need the machine.

A lot of human history, he says, is the work of making other people *other*. On a network the other person is bits, which makes contempt cheap. Put two people in a room and two constraints arrive at once: you have to face their humanity, and they might punch you if you go too far. "It's hard to troll someone when you're looking them in the eye."

Isbell's group is the Lab for Interactive Artificial Intelligence. He is, by training and by first paper — published when the conference was still called NIPS — a machine-learning person who believes in data and statistics. Machine learning is not the destination. He wants an intelligent artifact, as interesting as a human being in its own way, and he does not expect to live to see it. Learning is required for intelligence, so he has to figure out learning. Interactive, in the name, is the reminder that the goal is larger than a better learner.

He treats ML as engineering against a task and AI as the more philosophical project of true intelligence, often as a way of understanding ourselves. Asked whether that intelligence is one system or a society of them, he says interaction is fundamental to *our* kind of intelligence. He does not care whether a tree makes a sound when it falls and no one is around, because without an observer it does not matter. We are intelligent with other people. Isolated cleverness is, in some sense, a meaningless act. We even hard-wire intention onto objects that do not have it.

That stance has a practical edge. You cannot assume the other party saw the clock on the wall, so you spend enormous time communicating, and you project your own reasons onto them in order to predict what they will do. Isbell, facing a new Chinese restaurant, orders General Tso's chicken and hot-and-sour soup because that is the easy choice. Other people order chicken and broccoli. He does not understand them; they do not understand him; they still have to talk, and both of them will drift. Interactive AI is not learning a task. It is adapting, over a long horizon, to someone who is changing — including the version of you from fifteen years ago.

Fridman notes that Alexa-class systems still know almost nothing about you in the temporally thick way people know each other — the crumbs you pick up over years. Isbell's reply is blunt. The machine-learning community has not spent much time on lifelong learning, or even on learning in that sense, because everyone is task-focused and overfitting the problem in hand. Transfer learning still begins by inventing a task. It is looking for the keys under the lamp because that is where the light is.

The target keeps moving. AI, he jokes, is the art of making computers act the way they do in the movies, and they keep releasing new movies. The moment you understand a trick, it stops counting as AI and becomes take-the-derivative, divide-by-two, and a sliding window. Incentives in academia and industry reward the next paper and the next product, so people hill-climb. Even the researchers famous enough to take a huge risk have spent a career taking small ones. His prescription is to stop solving equals-one. Build something that lives for months in an environment you cannot control, accept that it will take five years, and quit rerunning the same experiment to shave a point off a metric.

What about just making the model huge — GPT-3, all of YouTube, steroids of compute — until it swallows reasoning, painting, music, and love? Isbell cannot think of a more terrifying world than one founded on YouTube videos. You will learn something, so the experiment is worth doing. You will not get there. The subtler failure is that we say we want systems to learn, and when they do, they change *us*. We redefine success. Google is his exhibit.

Information retrieval crawled for decades on vector-space models from a book Isbell dates to around 1971. Then Google made ad hoc search look easy. The problem had changed. When there are literally ten million answers to the query, "give me relevant results" is no longer the job. The job is "don't give me anything irrelevant, at least on the first pages," because nobody looks later. Google, on that search box, is not solving retrieval. It is minimizing false positives, which is not the same as getting an answer. Users also trained themselves to type the keywords that summon the page they already wanted, then congratulated the machine for understanding them.

Faculty hiring, he says, is the same algorithm. Departments claim they want the best assistant professor. In 2017, about 60 percent of faculty at the top four computer-science programs — MIT, Stanford, Berkeley, Carnegie Mellon, a field with no top five, only a four-way tie for first — had earned their Ph.D.s from those four. In the top ten, roughly 65 percent came from the top ten. Across the top 55 ranked departments, half the faculty came from the top ten. There is no universe in which most of the best faculty, even at R1 universities, come from ten places, especially given how few Ph.D.s some of those schools produce.

In the 1980s nobody got fired for buying IBM. Nobody gets fired for hiring an MIT Ph.D. If that person is terrible, the pedigree is the alibi. The same person from a school you do not respect is a risk you took. You hire one or two people a year and may keep them for thirty, so every incentive points at minimizing false positives: you do not care about saying no wrongly, only about saying yes wrongly. If a beautiful answer sits on page thirteen, search does not care. Fridman asks how you ever find the Grigori Perelman — the awkward mind who will not play departmental etiquette and might put "machine learning is a waste of time" on the CV. Inject a little randomness, Isbell says, the way you would in a Bayesian search. With people the cost feels too high, because a bad hire also teaches students.

So he works one level back: enlarge the pool. Informal data from friends, later a real study, says graduate admissions are simple. You admit almost everyone from schools ranked above you, most from schools near you, and almost no one from below, except unranked liberal-arts colleges such as Harvey Mudd. That means whether you become a professor at Cornell was largely decided when you were seventeen. U.S. News computer-science rankings, unlike the magazine's other lists, are 100 percent reputation: two people per unit score everyone. Georgia Tech is the only computing organization that climbed from outside the top twenty into the top ten, which Isbell traces to visible bets — a college of computing at a public university, a redesigned undergraduate program in a crisis, an online master's when others were still arguing about MOOCs. Reputation moves when peers look up and say those people are better than we thought. The rest is inertia.

Fridman says rankings feel like a narrative space, not something grounded. Isbell will not give him the contrast. Narrative is objective, he argues — an objective measure of everybody else's opinions. It is not physics. It is still a measurement. Tell him something that is actually objective, he says, and he offers cameras.

Cameras, which are just light on film, did not work well for dark-skinned people until around the 1970s. The optics were not a mystery. The buyers, in the United States and Western Europe, were relatively light-skinned, so the engineering privileged that part of the spectrum. What forced better film was not a sudden moral awakening. Furniture makers needed pictures of mahogany. Candy makers needed pictures of chocolate. Every so-called objective instrument is a pile of tradeoffs made by people: this band of the spectrum or that one, this cost or that customer. When a conglomerate shows up with money, the tradeoffs move. Even objectivity ends in a story, because otherwise it is not engineering.

The lab is also called P-Funk. Isbell's quirk is that he hears words and tries to make acronyms of them, then changes the words if they will not cooperate. P is probabilistic, because the work is uncertainty; the rest of Funk he leaves to each student. When they graduate he hands them a hat, star glasses, a P-Funk medallion, and a pair of fuzzy dice. Fridman, born in the Soviet Union, asks for an education in this American thing. First correction: hip-hop and rap are not the same. Hip-hop is a way of life — graffiti, tagging, popping, locking, the DJ's scratching. Rap is the music inside it.

The way in is whatever the listener already likes. For Fridman he would start at Led Zeppelin, then Public Enemy's *It Takes a Nation of Millions to Hold Us Back*, which Isbell calls, without much hedging, the best album ever made. Rick Rubin produced it; the music had an eighties heavy-metal grain; the lyrics were political. Then Mos Def, then back to EPMD, then to The Last Poets' first album in 1970, so you can see how long the line is. Confusion about a genre, he says, is the same problem they have been discussing all hour: narrative, membership, immersion. Jazz works that way. One day Charlie Parker clicks because you have been soaked long enough to use words nobody else understands.

Sampling is not a modern trick. The DJ used to get top billing for assembling records into something you could dance to. Run-DMC's Jam Master Jay would record himself on piano and sample the tape rather than play the piano live — thinking in pieces, assembling new music from fragments, which is also what jazz does with a spin on top. Fridman mourns popular rap drifting from lyrics toward the beat. Isbell came in through lyrics too; he wrote his last review the month he graduated. Every time he despairs of the charts, someone points him to an underground track and he is whole again. He adds Junior Wells, Chicago blues, *Snatch It Back and Hold It*: James Brown before James Brown was James Brown. Funk is a continuous line that only got its seventies name later. He does not reject disco either.

A tool is a hammer. A skill is knowing how to swing it. A discipline, Isbell says, is a mindset — a fundamental way of seeing the world. Physicists discover with the scientific method. Mathematicians build abstractions and look for steady-state truths inside them. Engineers build demonstrably better artifacts under tradeoffs. Computing has to earn a fourth seat by having a mindset of its own.

What distinguishes a computationalist is the understanding that models, languages, and machines are equivalent. Regular expressions and finite-state machines are the same object, one wearing the face of a language and one wearing the face of a machine, and that equivalence leaks into debugging. Because the machine is executable, the triangle is dynamic. Mathematicians, when he writes for them, demand that he freeze it. In their world truth is a thing, not a process that keeps running. Computing brought self-reference and running into the picture. Ten or fifteen years after he put the triangle on paper, he added the missing vertex: a person. Data and compute got large enough to do striking things, but the interesting part is human beings inside the loop, intelligence interacting with the models, languages, and machines and giving the information meaning.

Everyone does not need to learn to code. Everyone needs to think with a kind of precision he defines as repeatable: describe a problem so a machine can execute it. Humans are bad at this. We spend most of conversation vaguely guessing at meaning until it is good enough to act. You cannot do that with machines yet. Coding is a crude means. The idea of a programming language — that you fiddle with parameters and the outcome changes — is the lesson. A hole in current machine learning, he argues, is that people get lost in the deep network and forget they are doing software engineering, making decisions on parameters they invented. Computing education should force those assumptions onto the table. Call them parameters or if-then statements; the point is you have to be explicit, because one day the model will run. Slightly better social-media code will not fix social media. It only works if users understand what is being done to them and take control of their own behavior.

Isbell was born in Chattanooga. His earliest memory is arriving in Atlanta on a moving truck at three and a half, so he claims the city. The Atlanta of the 1970s and 1980s was deeply segregated, though not always obviously so if you were young or not looking. Highways carved race and class. He grew up on the poor side of a Black area and was very much aware of race. In first grade he had a girlfriend, a white girl named Heather; they and her friends had a serious talk about whether a Black boy and a white girl could be boyfriend and girlfriend, and decided they could. In first grade that meant extra recess and one Eskimo kiss that felt scandalous because everyone was watching.

Awareness of race is not the same as awareness of being a minority. At six, the world is the seven people you see every day, and in that world he was not outnumbered. Atlanta was home of the civil rights movement. The news anchors on the handful of channels were Black. The mayor was Black, and always had been. Then he went to Georgia Tech. Walking from west campus to east on the first day, among hundreds of students, he was the only Black person he saw. It was enlightening and off-putting, and it continued for much of his career. College also did what college is supposed to do. In Atlanta you were Black or white. At Tech he met students of Asian descent and students who would later be called Hispanic.

He applied to one undergraduate school, Georgia Tech, because he was, in his word, stupid — no one told him there were other options. For graduate school he tried three: Georgia Tech, MIT, and Carnegie Mellon. After the MIT offer he asked twenty people whether they liked Boston. Ten loved it and ten hated it. The ten who loved it were all white; the ten who hated it were all Black; the two stories were almost the same inside each cluster. White friends talked about coffee shops, Harvard Square, students, walking. Black friends talked about which parts of town you did not want to be caught in after dark. He understood both on arrival. In 1990 a major metro area had not a single radio station playing what he calls Black music. Atlanta had an economically viable, socially cohesive Black middle class. Boston, where it existed, was scattered, and the concentrated Black neighborhoods tended to be poor.

Ninth grade had sent him to a public magnet, the Math and Science Academy: 385 students, all but four of them Black, including a former mayor's daughter and a cousin of Michael Jackson. The parking lot was a problem because the cars were not poor kids' cars. You could be Black and rich or Black and poor, and both were visible. Boston did not feel that way. It felt like the interracial-dating capital of the country and also like a place where everyone knew which train you should not ride and which house you could not live in — greater Boston, 1992. Inside MIT he always felt empowered. Some of that, he says, was his own delusion, but it worked. People in power wanted him to succeed. He names Jackie Robinson syndrome: the first one has to be perfect, because if that person fails, nobody comes after for a long time, so it is in everyone's interest that the first one makes it. Shirley Jackson had earned MIT's first Black Ph.D. around 1971; the first Black undergraduate was still about eight years away. The university, as he experienced it, was not the enemy. The rest of the city's institutions felt designed to owe him nothing.

Fridman asks whether the lesson is to ignore the small forces around you and always see yourself as empowered. Isbell will only go halfway. You have to be deluded enough to finish a Ph.D., which requires believing you can invent something no one else has. You cannot be so deluded that you fail to step out of the way of the oncoming train. He was lucky in friends, in his mother, in family that long-distance calls — then an expensive thing — could still reach.

He has been pulled over more than once. Past fifty, he still cannot see a police officer without getting very tense. Halfway between Boston and Wellesley, an officer drew a gun. The thought in Isbell's head was that if the man shot him right then, he would get away with it. Years later a worse thought arrived: in today's world he would get away with it *and* Isbell would be painted a villain, big and scary, moving too fast, failing to do as he was told — and he had not done as he was told. That stains a family, a legacy, a history. He does not think the officer went out that night to kill anyone, and would not be surprised if the man never drew a gun again. He cannot remember the name. He remembers the gun. Guns, in your face, are much larger than you think. The stop was supposedly a red light Isbell does not believe he ran. He may not even have gotten a ticket.

Fire, he says, is necessary, and you must respect it because it will burn you. A cop is a giant ball of flame. Some families roast marshmallows beside it; he does not roast marshmallows. Of course that stance can become hate. Hate might even be the logical endpoint. You still have to live in the world, and hate takes too much energy to carry every day. Reserve it for when it is useful. He will empathize with what police officers go through. He will also cite two numbers: the job is not even among the ten most dangerous in the United States — you are more likely to be killed in a taxicab — and nearly half of officer deaths are suicides, which means something is happening inside that cluster. Step up a level and the problem is a narrative structure that makes it easy to sit in one cluster and forget that the people in the other are like you. If you had grown up in that house, as that person, as a woman, would you believe what the "crazy" person believes? Probably yes, because they believe it, and fundamentally they are the same as you. The exercise solves nothing. It knocks a little off the wall.

How do you fix Twitter, Facebook, racism? Isbell calls them structural. Individual conversations matter, but you have to build structures in which those conversations can happen all the time, relatively safely, so people can see that others have had different lives and are still the same kind of creature. He knows it sounds sweet. Optimism, for him, is one of the necessary delusions. The hard part is that changing the infrastructure is in almost no one's interest right now. Why this moment? He prefers why not earlier. You need a generation old enough to be angry and not so old that it remembers the last time. There were civil rights movements in the 1930s and 1940s; they had a different character. His guess, offered with a historian's apology, is television. Martin Luther King Jr.'s early effort in Georgia failed when a sheriff quietly loaded people onto trucks, jailed them far away, and gave the press no reason to stay. Next stop was Birmingham, Bull Connor, fire hoses on children, outrage in the living room. Part of the delusion is pretending nothing bad is happening. Once it is in your face, ignoring it gets harder. Social media is that visibility again, turned up, plus demographic change fast enough to unsettle some people and not so fast as to produce a total backlash.

Visibility is not a guarantee. Before Rodney King, Isbell remembers a police officer who told a news crew that Southern California was ugly, invited them to follow him for a week, and on the first night — crossing into a city, the other cops unaware he was one of them — had his face shoved into a glass window on camera. It ran, then vanished in two days. Isbell sometimes thinks he dreamed it. Rodney King went as viral as 1991 allowed, and then came April 29. He cannot name the difference except that belief and evidence now feed each other, with phones and body cameras in the mix. Go read the op-eds after Rodney King that said violence is not the answer, then the ones from the 1980 riots: the same words, enough that he is surprised nobody was accused of plagiarism.

Fridman offers the usual fork, King versus Malcolm X. Isbell first corrects the ages: Malcolm X was older; people remember him as the young angry one because he died sooner, which is a narrative device. Then he rejects the choice. Without taking a moral position, he will say the annoying fact: violence has worked. It is a shout. People far more talented than he have said violence is the voice of the unheard — what you do when you feel you have no other option. Sometimes we think they are justified and sometimes we do not; either way, when you shout, people hear that you are shouting even if they miss the words. A less provocative version, which he thinks is true: all change that touches power requires struggle. The struggle need not be violent. The powerful do not give power away. The same, he adds, is true of understanding calculus.

Will everything be all right next year? He tends to think so because he is the sort of person who thinks so. His mother has always said this too shall pass — not only the bad. Everything passes, including a sixteen-year-old daughter who will soon ignore him the way he ignored his parents. He is not worried that AGI will end us; at any moment something might, and excluding the things we cannot control, he likes our odds. If an asteroid is coming, there is little he can do, so he assumes it is not. It makes food taste better. In the 1980s, before Google, he assembled a This Week in Black History calendar from whatever he could steal, and what stuck were inventors in the lineage of Benjamin Banneker, making things we still use while the world around them could erase them for being in the wrong state at the wrong time, often without credit. Immortality, a line he borrowed, is doing something — preferably good — so your great-grandchildren know your name. He does not know if he is afraid of death. After a thyroid surgery he had scheduled for December 19 so he could say he made it to forty-nine, five hours vanished with no sensation of time passing. He thinks that is death. He is soothed that he will not notice, and furious that he will not notice. He would rather find out whether immortality sucks, and choose. Asked for the meaning of a life whose great-grandchildren will not know his middle name, he offers another borrowed sentence: it is well worth remembering that the entire universe, save for one trifling exception, is composed entirely of others.