Without Analogies, There Can Be No Concepts

Lex Fridman opens with the name of Mitchell's book, *Artificial Intelligence: A Guide for Thinking Humans*, and with the name of his own show. He asks the old question about roses: what does she think of the term that labels the whole field? Mitchell is not crazy about it. It means too many different things to different people, and *intelligence* itself is barely defined — too many kinds, degrees, and approaches to pin down.

John McCarthy coined *artificial intelligence* in 1956, she says, partly to differentiate the work from cybernetics, the related movement of the time. From what she has read, he later regretted the choice. Herbert Simon pushed for *complex information processing* and lost. That alternative is probably equally vague, Mitchell admits, but it would have attracted a different crowd. She was drawn in by the phenomenon of intelligence; if the field had worn Simon's name, she might be doing something else.

The trouble is not only the word *intelligence*. *Cognitive systems* carries its own baggage, because people like to split cognition from perception, a line Mitchell does not accept. When she left graduate school in 1990, during an AI winter, she was advised not to put *artificial intelligence* on her CV at all. The euphemism of the moment was *intelligent systems*. People have spent the whole history of the field trying to say what they are actually talking about, from John Searle's strong versus weak AI to today's *artificial general intelligence* and Yann LeCun's preference for *human-level intelligence*.

Early on, a lot of people thought you could not play chess without general human-level intelligence. Once computers played it better than humans, that view was revised: maybe chess was never the test. Mitchell says this has been a theme throughout the field. Once a machine can do a task, we look back and say that changes our understanding of intelligence, because we do not want to call the machine intelligent.

Will the line keep moving forever? She does not see any reason, in principle, why we could not create something we would consider intelligent, though she does not know how we would be sure. Our own view of intelligence may have to be refined until we finally know what we mean. The machines that get there may not look like the ones we have now. One side effect, she thinks, will be a clearer view of our own machine-like qualities: cells are mechanical, they run algorithms and process information, and somehow out of a massive number of them we get the emergent property we call intelligence.

If she had to bet, creating intelligence without understanding our own minds to a significant extent is not possible. That is still a big open question. She has been surprised at how far brute-force approaches based on big data and huge networks can go; she would not have expected it, and those methods have nothing to do with how our minds work. Computers are already smarter than us at multiplication and routing. What would actually usurp someone's sense of humanity is personal. For her former adviser Douglas Hofstadter, a program that created beautiful music would be terrifying, because that sits at the core of being human in a way speech recognition does not.

The field has been terrible at predicting its own future. Mitchell thinks that will improve as it matures, and that the root failure is how little we understand our own intelligence. She confirms the famous story: Marvin Minsky assigned computer vision as a summer project to undergraduates. The proposal, which everyone should read, basically lists the fundamental problems of computer vision, many of them still unsolved. Minsky was extremely smart and a sophisticated thinker. He still underestimated the job.

The reason is specific. The things we do are so invisible to us that they do not feel like work. Looking out at the world and describing what we see is immediate; it feels like no effort at all. Underneath, a huge amount of unconscious computation is running, and that is why we overestimate how easy it will be to get computers to do it. Mitchell quotes someone in her book who said human-level intelligence is a hundred Nobel Prizes away — a useful unit, she likes, because it means that many major discoveries still have to be made. There is, of course, no Nobel Prize in AI. Not yet.

Asked for a number, she says more than a hundred years. Her guess is that the path has to go through understanding our own cognitive systems, not through taking today's machine learning and scaling it exponentially with hardware and data. Current approaches will get better at narrow AI, but she thinks they have fundamental limits, including weaknesses that come from supervised learning and feed-forward networks. Fridman is torn. He agrees, and he is also more and more surprised by how far the methods go; he thinks they will impress people further than skeptics admit, while human-level intelligence remains much farther than anyone realizes. Mitchell grants that the methods are more powerful than people credit. She still does not think the approach is sustainable.

The surprise, when she was writing the book, was how far apart people in the field are on how far it has come and what happens next. One group is the singularity and transhumanist camp: we are on the steep part of an exponential, and in the next thirty years we will see superintelligent AI and upload our brains. Most people who work in AI disagree, she says. Another group skips the singularity but believes current deep learning will scale all the way to human-level AI. Quite a few of them work in the AI groups at big tech companies, and they tend to think we are not that far.

Yann LeCun is a third pole. He still expects Nobel-level breakthroughs, but he thinks they will be built on top of deep learning. Supervised learning is not sustainable; unsupervised learning is the key. Mitchell thinks that is probably true, and that unsupervised learning the way humans do it will be harder than people think. Gary Marcus represents a hybrid view: deep learning is one module, symbolic methods have to come back, and nobody knows how to join them — which layer is the foundation and which is the icing. Then there are the causality people, who say deep learning as formulated today has no notion of cause and is therefore doomed unless it is given one.

A cognitive-science crowd pushes developmental learning, how babies learn, and intuitive physics. Someone quipped that machines also need intuitive metaphysics: objects exist, causality exists, the sort of things we may be born with. A system that sees a pile of pixels and gets ten million examples still may not learn that there are objects in the world. These views are not mutually exclusive; LeCun, she notes, agrees on development, causality, and intuitive physics, but still thinks learning can take us all the way without much innate structure. Mitchell herself is sympathetic to the cognitive-science side, which is how she entered the field, and she has become more of an embodiment adherent: without a body, it will be very hard to learn what we need to learn about the world.

In 1984, as a new graduate student, Mitchell joined Douglas Hofstadter's group and started work on Copycat, a program that makes analogies in an idealized world of letter strings. Hofstadter's view, which she cites from *Surfaces and Essences*, written with Emmanuel Sander, is that analogy is not the shoe-is-to-foot puzzle on an IQ test. It is a core aspect of thinking, pervasive in language, thought, and perception. Without concepts there can be no thought, and without analogies there can be no concepts. Mitchell has said that line should be one of the mantras of AI, and she adds her own: how to form and fluidly use concepts is the most important open problem in the field.

Copycat was an agent-based system inspired by old blackboard architectures. Little detectors posted findings in a shared workspace, marked groups such as a run of successors in the alphabet, and built connections among them. Given *ABC* changes to *ABD*, what does *DEF* change to, the program needed the alphabet, the idea of a letter's successor, and a concept of sameness already built in. All of its concepts were innate, a serious limit they accepted on purpose. The question was not how concepts form from scratch, but how, if you have some, you apply them flexibly to new situations.

A concept, she says, is a fundamental unit of thought, embedded in a space of other concepts at varying distances, and people form new ones as complex combinations all the time. Analogy is recognizing that one situation is *essentially* the same as another, with *essentially* as the keyword. Three days earlier she had done a podcast interview in Washington. The person, the microphones, the questions, and the building were all different, yet it was the same kind of event — and *podcast interview* was itself a new concept for her. A friend who says "the same thing happened to me" about a completely different situation is making an analogy without even using the word.

She pushes the claim until it covers ordinary perception. Every time she recognizes that Fridman is a person, that is analogy: she has a concept of person and is applying it to him. The same is true of seeing someone walking a dog; the details are always different. Perception takes raw sensory input, integrates it into an understanding of the world, and updates that understanding, and the whole process is a giant mess of analogies. Engineering that, she thinks, comes down to internal models. Some psychology treats concepts as mental simulations that generate predictions and expectations. Those models, in a brain, are densely interconnected.

Fridman asks how big the network of concepts in a human head would be if you tried to draw it. Mitchell does not even know what units to use. Something on the order of a hundred billion neurons, trillions of synapses, chemical as well as electrical encoding, firing rates nobody fully understands: the capacity looks enormous. What we know is not a pile of facts. It is integrated into a whole we can make analogies with. That is why she keeps her distance from the old dream of turning Wikipedia into a knowledge base for analogy. People have been building semantic webs, expert systems, and projects such as MIT's ConceptNet for a long time. The problem of common sense is that most of our knowledge is invisible to us. It is not in Wikipedia.

If you built a site for intuitive physics and intuitive psychology, would it be bigger or smaller than Wikipedia? It depends, she says, on whom you are describing it to. Representation is the real difficulty. She can write down F=ma and Newton's laws, and a lot of physics can be deduced from that, but that is probably not the best form of the knowledge for the reasoning we want a machine to do. Doug Lenat's Cyc project tried to encode all of common sense, including the invisible part, in a logical representation, and she does not think it did the things he hoped. She admires him anyway. He was one of the earliest people to say that expert systems would not get you to AI and that common sense was the real target, and he gave up an academic career to pursue it. The approach itself, she thinks, will not work — unless the history books later say Cyc found its breakthrough in 2058.

On hardware she is less mystical. Turing computation is probably sufficient; she sees no evidence for Roger Penrose's claim that intelligence requires continuous values and quantum mechanics. We have invented the machines we need. We have not invented the right algorithms and architecture, and she does not think we can scale up current ways of programming them. The promising direction in her own work is active generative models: a mental simulation that produces predictions, tells you where to look, and mixes with bottom-up input until the blend *is* perception. Seeing someone riding a bike while holding a leash, she can map it onto dog-walking, notice the differences, and guess they are out for exercise rather than commuting. Deep networks can build hierarchies of features, in the spirit of Hubel and Wiesel, but they apply the same filters to every part of an image. They lack the conceptual expectation that says this region is irrelevant and that one matters.

Fridman asks whether brute-force learning can go all the way, the way birds might criticize airplanes for not really flying. Mitchell does not think so. It is an open question, but she tends to be on the nativist side: some things we were evolved to learn, and learning cannot happen without them. Her exhibit is DeepMind's Atari work. A system learned Breakout from screen pixels and played it a thousand percent better than humans. It even found the trick of tunneling through the side of the brick wall so the ball could bounce off the ceiling and wipe out bricks.

Another group then moved the player's paddle up a few pixels and asked a trained Deep Q-learning system to transfer. A human has no trouble. The system could not. Mitchell's point is not that the original result was unimpressive — she calls it incredibly impressive — but that the agent had not learned the concept of a paddle, or of a ball, or of tunneling. It learned something. Watching it, we anthropomorphized the behavior and described it in our terms. Because those concepts were not there, transfer failed.

Fridman objects that moving the paddle also injects an anthropomorphic assumption, then scores the system for failing it. The Atari result, like AlphaGo and self-play, still surprises him at a deep level. Mitchell does not deny the surprise. The question she is asking is whether these systems learn the concepts humans consider relevant. Train on paddles in many positions and maybe it would learn that one; she is not sure. Scale the same method up to more complicated worlds, and to what extent can a machine that only gets raw data divide the world into relevant concepts? She does not know, but she would bet that without some innate structure it cannot. Ten years earlier Fridman would have agreed one hundred percent. Self-play has left him with a glimmer of hope. Mitchell grants that self-play is amazingly powerful, a line you can trace to Arthur Samuel's checkers program.

Fridman works on autonomous vehicles, and Mitchell uses driving as a case of a problem that looks easy until you try it. The world is open-ended. In normal conditions, with decent weather, current cars do well. The trouble is the long tail: so many things can happen that were not in the training data, and the system has no common sense with which to handle them. It is the paddle, moved, at city scale. Her reading is that current vision systems cannot tell which obstacles they should stop for, so they slam on the brakes, and the most common accident is being rear-ended by a human who did not expect the stop.

Fridman splits the problem into perception and policy. Many Level 4 vehicles with safety drivers — Waymo, Cruise, and the like — are extremely conservative. Their policy overreacts to anything that might be an obstacle, which reads as unpredictable to the humans around them. Human drivers are cautious too, but they are also in a hurry and sometimes angry, especially in Boston. He adds that companies often advertise deep learning partly to recruit, while lidar and other sensors still do a lot of the reliable detection. Tesla's bet is vision-first. Elon Musk treats lidar as a crutch; the camera contains more information if you want to learn. The fallback sensor then becomes radar, which is weak on stationary objects — the kind of gap behind crashes into stopped fire trucks.

Tesla's data pipeline, Fridman says, is genuinely impressive: active learning on edge cases, and multitask learning that breaks driving into dozens or hundreds of smaller problems. He has grown less skeptical about how much of driving can be learned that way, while still thinking the timeline is much farther than the CEO claims. Mitchell's practical judgment is that dealing with any situation requires full human intelligence. Human accidents are mostly inattention, alcohol, distraction; the machine is always paying attention and fails for not being smart enough. Autonomous vehicles will eventually be safer than humans, and that is a low bar. What she expects is not sleeping in the back seat on the way anywhere, but instrumented areas with sensors and maps where cars can do fine without common sense — as long as pedestrians do not mess with them too much. The nonverbal dance among drivers, cyclists, and walkers is knowledge humans have and cannot articulate.

Mitchell is coming around more and more to the idea that human-level intelligence needs a body, though she is not sure. Fridman draws a distinction he associates with LeCun: grounding, meaning interaction with reality, versus a body you have to preserve, that can die, that can be afraid. Mitchell has not thought it through in those terms, but she guesses both. Intelligence is hard to separate from the desire for self-preservation, from emotion, from the non-rational stuff that seems to get in the way of logical thinking.

A huge part of human intelligence is social. We evolved to deal with other people, and that is so ingrained that it is hard to cut intelligence away from it. For seventy years, more or less, AI has done the cutting anyway. The Cartesian assumption was that we are trying to create a thinking thing and need not care about the rest. Mitchell thinks the rest is fundamental. Emotion is not an obstacle to intelligence. It is an integral part of how we perceive the world, make decisions, interact with others, and understand them.

To guess what Fridman is likely to do next, she needs a theory of mind, and that is largely a theory of emotion, motivations, and goals. She understands those by simulating them in herself, the sort of system people associate with mirror neurons. She cannot prove the machinery is necessary. It seems very likely. Fridman, who is Russian, jokes about a Slavic-pessimism argument against superintelligence that was going around: nothing ever actually works. Mitchell laughs, then answers the serious version of the question. The limits we treat as bugs — fear, suffering, mortality — are part of how human intelligence works.

Mitchell's *New York Times* op-ed, titled by the editors *We Shouldn't Be Scared by Superintelligent AI* — a headline she would not have chosen — was a reply to Stuart Russell's earlier column summarizing *Human Compatible*. Russell's argument is that if we have superintelligent AI, its values must be aligned with ours, and it has to learn what we really want. His example: ask it to solve climate change, and it decides the best way to lower atmospheric carbon is to kill all the humans. To Mitchell that made no sense. A system superintelligent at climate science cannot be too stupid to know that humans do not want to be killed. You do not get one of those abilities without the other.

She is equally unconvinced by Nick Bostrom's orthogonality thesis, the idea that a system's goals and its values need not be aligned. The paperclip maximizer, as a thought experiment or as something that could be built, does not work for her. Intelligence is more complex than these presentations allow. Rationality, values, and emotion are not separable dimensions. The picture of a machine that is superintelligent along one axis and empty along the others is what she was trying to criticize.

Fridman reads a careful response from Yoshua Bengio: cognitive biases may be tied to learning, but value alignment is already a problem in the form of super-powerful companies whose objective functions are not aligned with humanity's well-being. Mitchell agrees only halfway. Corporations are people; the values are the values of the people who run them. Fridman notes that the algorithm can still steer a mass of humans, as when an ad-driven engagement loop pushes a company toward more division. The question she wants asked is who has the agency. A facial-recognition net that is worse on darker skin is not itself a racist agent; the agency sits in the training data, the cameras, and the use. Russell's claim is different in kind, not just degree: the machine itself now decides, and it is the thing that has values.

Do Musk, Russell, and Bostrom have a point about existential risk, even if the date is unknown? Possible, she says, but there are many closer existential threats. Her timescale is more than a hundred years, maybe more than five hundred. In that span, a million technologies we cannot now predict will change society, and the AI risk we discuss then will not mean what it means today. Nuclear weapons, climate, poverty, possible pandemics — those are in the queue. AI existential risk should not be the first priority. She is not criticizing Russell or Bostrom for worrying; some people should. Her target was their picture of intelligence. Fridman adds that it is good for people to believe different things, and that the work remains useful even when they fight over the word *superintelligence*. She agrees, and she is glad the *Times* will publish the fight.

What would count as a good test of intelligence? Mitchell still thinks the original idea of the Turing test is a good one; she cannot think of anything better. The versions actually run so far have been impoverished. In her book she describes a bet between Ray Kurzweil and Mitchell Kapor: by 2029 a machine will pass a Turing test with specified hours and expert judges. Kurzweil says yes, Kapor says no. Speaking in 2019, they had about nine years left. If a machine passed that test, she would be willing to call it intelligent. People would then say it was just a language model. Fridman asks whether a long conversation would be enough for her. Probably, she says, because carrying one would require a deep commonsense understanding of the world, and a long enough conversation would reveal whether that understanding was there.

Complexity, like intelligence, is an overused word. Mitchell's book on it is about complex systems: nature, technology, and society in which emergence appears. A brain has billions of neurons; one neuron is not very complex compared with the whole. Interactions and dynamics produce what we call intelligence or consciousness. The field looks for general principles under systems like that. Emergence usually sits on simple underlying interactions, in large numbers. Reductionism — divide a system into cells, atoms, or particles, understand the parts, then sum them — works for most of science. It does not always yield the things we most want to understand. You will not see intelligence by staring at a single neuron.

The Human Genome Project is her cautionary example. The excitement of sequencing was a reductionist hope: list the parts, then find which gene causes which disease. The parts do not cause the outcomes of interest. The networks of interaction do. Asked for the most beautiful complex system she has been captivated by, she names cellular automata, which she worked on for several years. That such simple rules on a grid can produce seemingly unlimited complexity still surprises her. It is humbling and awe-inspiring, still mysterious, and also hopeful: maybe you can engineer complexity from very little at the start.

The Santa Fe Institute, in Fridman's mind, is a mystical place where brilliant people study the edge of chaos. Mitchell says that is about right. It was founded in 1984 by scientists, many from Los Alamos, a forty-minute drive away, mostly physicists and chemists frustrated that their fields were not approaching big interdisciplinary questions. The names are heavy: George Cowan, who had been on the Manhattan Project; Nicholas Metropolis; Murray Gell-Mann; Kenneth Arrow, a Nobel economist. They started workshops, and the institute grew. It has lived on the edge of chaos itself, with no large endowment, running on donations, grants, and business partnerships. A handful of resident faculty, on five-year terms, sit at the center; around them are postdocs and on the order of a hundred external faculty, including Mitchell and the physicist Sean Carroll, who visit for stretches of time.

For people who want in, there is a four-week Complex Systems Summer School, specialty schools on computational social science and on climate and sustainability, short courses, and an online platform that includes Mitchell's own introduction to complexity. Technical seminars, colloquia, and public community lectures all go on YouTube. The lesson she still carries from Hofstadter is more specific. Faced with a complex problem, idealize it as much as possible until you can see its essence. That is how Copycat was born: take analogy-making and make it as idealized as possible while keeping the thing you actually want to study. Hofstadter's physics doctorate shows in the method. Early AI worked in microworlds such as blocks world, then was told those would not scale, and turned toward messier real-world problems. Now there is a return, even to blocks world itself. The fundamental challenges of intelligence may live in those idealized problems more than people realized.

Looking back, the work she is proudest of is Copycat. It is different from almost everything else people have done, and there are still ideas in it waiting to be explored. One of the happiest days of her life, aside from the births of her children, was the birth of Copycat: the moment it started making really interesting analogies. Fridman says she gave it life. Artificial life, she answers. For anyone who wants to pick it up, Hofstadter's *Fluid Concepts and Creative Analogies* describes the mechanism in detail; her own *Analogy-Making as Perception* is the book version of the dissertation; the code is linked from her web page, along with the successor project Metacat and a Python 3 implementation. The advice is plain: get the code, run it, play with it.