Slow Synapses Grow Fast Learners — Warmth Is the Harder Problem

Lex Fridman opens with a deceptively large question: how much of the human brain do we actually understand? Matt Botvinick, then director of neuroscience research at DeepMind, calls the present a weird moment in the field’s history. Researchers understand a lot at a high, very coarse level — what the organ is for, what kinds of computation it performs, which behaviors would have to be explained if you descended to mechanism — far more than when he was in high school. At the other end of the spectrum, new tools let people watch single neurons and even dendrites. Between those two views sits a yawning gap.

As he has gotten older he has become more reluctant to distinguish psychology from neuroscience. A nephrologist studying a kidney starts with what the thing is for: take blood carrying metabolites that should not be there, keep the good stuff, and excrete the rest as urine. Only then does the work become how. The brain, as far as he can tell, is for producing behavior: going from perceptual inputs to behavioral outputs that should be adaptive. That functional story is what psychology is about. The rest of neuroscience is figuring out how those operations are actually carried out.

Unlike the kidney, the jump from electrical signal to behavior is enormous, which is why he treats cognitive science, psychology, and neuroscience as one science. Fridman asks whether the stance is aspirational idealism or pragmatic engineering. Both, Botvinick says. He hopes and expects the framing to bear fruit over coming decades. He is also not sure what either field is doing if that is not the framing. He does not know what it means to understand the brain if part of the enterprise is not about the behavior being produced.

Fridman offers early astronomy as a counterexample: tracing planets without physics still had value. Botvinick agrees. You need Tycho Brahe’s data before you get Newton. He still treats that phase as a stage, not a destination. In graduate school he was puzzled by psychology’s “virtual causal mechanisms”: attention selects an object, then information about it is passed to the motor system. Those were metaphors. Nothing in the conversation reduced to a physical mechanism, which is what it would take to fully understand how behavior arises.

He soothed his nerves with the history of genetics. Mendelian genetics preceded Watson and Crick. For a significant stretch people productively investigated inheritance using what was essentially a metaphor of the gene: dominant, recessive, recombining. Where were the genes? They were an explanatory entity people had invented and loaded with causal properties. Later the blank was filled with a physical mechanism. The metaphor was still worth having, he says, because it gave a good sense of what kind of causal mechanism to look for.

Asked whether the interaction of neurons is itself a metaphor, he hesitates, warns that the answer might be controversial, then says no. Interactions among neurons are a physical fact — the place where causal interactions actually occur. You could descend into quantum events; he will not go down that rabbit hole. Remaining forever at psychology’s natural level of description would, for him, be disappointing. He wants to know how mental activity arises from neural activity. The converse is also true: studying neural activity with no sense of what you are trying to explain feels like groping around at random.

He would be the last person to say that psychology, on its own, is unproductive. He trained as a psychologist and is fond of saying he has learned more from psychology than from neuroscience. What most impressed him in graduate school was neuropsychology: analyzing people with different kinds of brain damage and isolating the specific deficit that arose from a lesion in a particular place. The experimentation struck him as creative, deliberate, good science. An experiment answered one question, raised another, and you felt yourself narrowing in on an approximate understanding of what that part of the brain was for.

He points to detailed studies of language — production, reception, the relationship among vision, reading, audition, and semantics. Beautiful models came out of that work and made you feel you understood something you had not understood before. He still agrees with Fridman’s complaint: the cost of highly controlled experiments is that, by construction, you miss the richness of the real world. What actually drew him into science was connectionism, the 1980s name for what is now called deep learning, at a moment when neural networks were used to model human cognition rather than to ship industrial products.

He went to medical school and also completed a graduate degree in art history. He was interested in surgery and in psychiatry, and he thought he must be the only person on the planet torn between those fields. His advisor, later revealed to be a famous psychoanalyst, said it was not uncommon: both are about going beneath the surface. On a psychiatry rotation he confessed he was sure nobody was doing the research he wanted — he did not yet have the word “cognition.” The advisor pulled down a still shrink-wrapped set of the PDP volumes, *Parallel Distributed Processing*. Botvinick took them back to the dorm and read them cover to cover.

Asked for the most beautiful, mysterious idea in the space of the brain, Botvinick names a paradox. The brain seems so distant, and yet it is responsible for the full transparency of everyday life. It is literally what makes everything obvious and familiar, and there is always one in the room with you. When he taught cognitive neuroscience at Princeton, the last thing he told students was that the usual metaphor of scientific wonder is looking at the stars. He is all for looking at the stars. His own sense of wonder comes from the extremely intimate brain.

He resonates with Fridman’s further point: the brain produces not only reasoning but the entirety of experience. Discussions of artificial intelligence often feel disappointingly narrow to someone who has always had an interest in art. There are many layers to full human experience, and it is not enough to say we will do cognition now and add emotion later. The scientific challenge of explaining that scope is overwhelming. It has always been his fixation.

Fridman is saddened by how much of language’s music and wit dies when a research community rallies around a benchmark. Botvinick agrees, then defends the first wave of deep learning models in cognition: those builders were focused on richness. As a graduate student he watched a dry-sounding debate on the English past tense. One camp said the mind encodes a rule plus a list of exceptions. Connectionists said look at the corpora. Most verbs take *-ed*, but the exceptions are not random; there are clues, and exceptions to the exceptions — quasi-regularity. You could try to write that structure in closed form. The right way to understand how the brain handles it, he still thinks, is to train a deep network on the data and see how it represents the richness.

Asked which gap between biological and artificial networks he would most want to close on the way to human-level intelligence, he names something many people in the field are also saying. Current systems lack the flexibility, the readiness to turn on a dime when context requires it, that is so characteristic of human behavior. He keeps a particular way of introducing Turing machines from an introductory computer-science course. A Turing machine is a machine emulator: it can emulate any other machine. That framing has stuck with him.

Humans are capacity-limited; they are obviously not Turing machines. They still have the ability to assemble behaviors very much unlike anything they have done before. Some basic mechanism implemented in the brain allows us, as he puts it, to run software. Is there magic, or is it just computation? He is happy to think of it as basic computation. He will not be satisfied until someone explains what those basic computations are that lead to the full richness of human cognition. Understanding the computations that let people do arithmetic or play chess will not be enough. He wants the whole thing.

On group behavior and the idea of a collective mind, Botvinick will not take credit for the multi-agent work coming out of his group or DeepMind. That human behavior arises within communities is, to him, self-evident. Fridman reaches for the bone in *2001: A Space Odyssey* and for Yuval Noah Harari’s claim that large numbers of people can hold an idea together — as in the pandemic shift from handshake to elbow bump, a distributed convergence with nobody in one room. Botvinick is open to the thought that a strictly bottom-up approach is wrongheaded, and that some aspects of intelligence can only be understood in groups. He has never been particularly convinced that intelligence inheres at the community level. He stubbornly defines it as an aspect of an individual human.

Fridman notes that this may be the reductionist dream of a scientist, and also that it may be a sad possibility: one isolated human would not become *Homo sapiens*. “That’s a possibility,” Botvinick says. What he will insist on is a different emphasis. A serious effort to understand human intelligence, or to build a human-like one, needs to pay as much attention to the structure of the environment as to the cognizing system, brain or machine. He took that from early study with neural-network pioneers including Jay McClelland and John Cohen. The structure of cognition is only partly a function of architecture and learning algorithms. What really shapes it is the interaction of those things with the structure of the world in which they are embedded.

Reinforcement learning makes the point obvious: you can only learn as much as you can simulate. DeepMind also made vivid the other agent as environment, through self-play, which Fridman calls one of the most exciting ideas in AI. Botvinick takes the moral in that narrower context: competition is essential for learning, at least there. A lot of what looks like intelligence can be a projection of environmental structure rather than a property of the system alone. When a model will not learn, the first place to look is often whether the world you gave it has graspable structure.

If he has an area of expertise, it is prefrontal cortex. The technical definition is anatomical, which he calls the coward’s way out. Motor regions are easy to identify; the cerebral cortex in front of them is defined as prefrontal. World War I opened the modern hunt for regional function, as soldiers returned with injuries. The research was productive and tragic. One of the frustrations facing neuropsychologists was that they could not pin down what damage to the most frontal tissue actually took away. Russian neuropsychologist Alexander Luria, still assigned to students of cognitive psychology, put a finger on flexibility: overriding a habit, doing something unusual, changing what you are doing from one moment to the next.

Later work sharpened a distinction between controlled and automatic behavior, or goal-directed versus habitual. The human brain has pathways dedicated to things you do all the time; they need to be automatic so cognitive capacity is freed. Think of learning to drive versus driving as an expert. Prefrontal cortex is now generally understood in contradistinction to that habitual domain. It lets you say: what I usually do in this situation is X, but given the context I should do Y. Reaching out to shake hands is probably habitual. Prefrontal cortex is what lets you bear in mind that something unusual is going on. Luria’s detection tests were of this form: I stick out my hand; present your elbow instead. A patient with frontal damage has great trouble. An offered hand elicits the handshake.

Are humans special? Do mice have prefrontal cortex? The question is timely because revolutionary tools for monitoring, measuring, and causally influencing neural activity exist for mice and fruit flies, not fully even for monkeys, let alone humans. People who study fruit flies will tell you flies are smarter than you think, able to learn and generalize in ways that suggest abstraction. Botvinick will describe mice taking forever on a task trivial for a human and conclude they lack the flexibility we care about. A mouse researcher will reply that the stimuli were unnatural; keep the logic, present the information in a form aligned with the animal’s habitat, and you might find more intelligence. Then come videos of a mouse dragging food home and clearing an obstacle. These, he says, are open questions.

There is overwhelming evidence of functional differentiation. It is clearly not the case that every region does the same thing. That follows immediately from the lesion work they have been discussing, and it is obvious if you stick an electrode in and measure neural activity. Look carefully, though, and the differences between regions are usually graded rather than discrete. It does not seem easy to divide the brain into true modules with clear boundaries and clean channels of communication. Prefrontal cortex itself is a bunch of subregions whose functions are not clearly defined and whose borders seem quite vague.

Recent work suggests that regions once thought tightly focused carry unexpected signals. Primary visual cortex is classically the first cortical station for vision, supposed to care about where the edges are in the scene. With enough data you can recover from it what behavior the animal is engaged in and how much reward is on offer. The history of neuroscience, Botvinick says, is an oscillation between the modular view and the big-mush view. He guesses the field will land in the middle, which is unfortunate for intuition: people find modules easy, and undifferentiated systems easy, and the in-between confusing. We will have to get used to it.

On how the brain carries information he is old-fashioned: the networks used in deep learning are a reasonable approximation. The usual story is a rate code — how quickly a neuron spikes — captured as a number. There is still uncertainty. Precise spike timing may matter. Computations inside the dendritic tree may be rich in ways that do not equate to anything in artificial nets. He still thinks they are getting somewhere by sticking to that high level of abstraction. Neuroscience 101 remains the working picture: transmitters arrive, receptors change voltage, a spike travels the axon. Artificial nets ignore individual spikes and treat frequency as “activity.” Asked whether the basic picture could be completely wrong, he does not hedge: “Oh heck yes.” He would not be a scientist if he thought there was no chance of being wrong.

From the 1980s there is an unbroken chain of a particular strategy. Train a multilayer network on a task given to a human, a monkey, or a rat, then ask whether units deep in the network resemble neurons deep in the brain. Over and over, typically with backpropagation, the patterns of activity look hauntingly similar. At a certain point coincidence starts looking unlikely to be meaningless. The circumstantial evidence is overwhelming. He stays open, of course, to flipping the table.

Their 2018 paper, “Prefrontal Cortex as a Meta-Reinforcement Learning System,” is about meta-learning: a learning algorithm that gives rise to another. A familiar version is learning a third foreign language after two. The first language is laborious; by the third you know you will have to conjugate and get through the sounds. A slow mechanism has updated a fast one. The paper’s origin was AI work in his group. They trained a recurrent network with standard reinforcement learning, not on one task but on a bunch of interrelated tasks, then handed it another task in the same family. A form of meta-learning spontaneously happened.

A recurrent net has memory in its activation patterns: loops of connectivity that let activity stick around and be updated, what neuroscience calls working memory. Those dynamics are shaped by synaptic weights, and the weights are being shaped by the reinforcement-learning algorithm. Train long enough on a sequence of two-button boxes, each with its own hidden reward probabilities, and the slow synaptic changes give rise to dynamics that themselves become a learning algorithm. Freeze the weights, tell the network it is done learning, and hand it a new box. It still figures out the better button, moving from exploration to exploitation in a rational way. No synapses moved. The activity dynamics had been sculpted over many boxes.

The group included neuroscientists, and bells went off: synaptic memory versus activity-based memory, and the heavy recurrence characteristic of prefrontal cortex. Maybe prefrontal cortex supports reinforcement learning strictly in its activation patterns — holding what you did and how much reward you got, and updating behavior from that. How did those dynamics get so smart? Perhaps the synaptic mechanisms believed to be mediated by dopamine shaped them. Fridman asks about stacks of meta-meta-meta learning. Botvinick says the idea can be overstretched; Jürgen Schmidhuber cheekily suggested it, possibly in a PhD thesis.

What fascinated him is that this flavor of meta-learning required nothing special. Many later algorithms are engineered: you decide you want meta-learning and write one. Theirs was not an algorithm. It automatically happens if a system has memory whose function can be shaped by reinforcement learning and is trained on interrelated tasks. You cannot stop it. An earlier paper he was involved with, on meta-learning in memory-augmented neural networks, used slot-based memory rather than recurrence and got the same story. Recurrence is not required; memory plus a task distribution is. The human world is saturated with that property. You never do exactly what you did yesterday, but everything has a family resemblance — endless variety with endless redundancy. Melanie Mitchell’s word for grabbing that resemblance is analogy.

Much of the group’s work takes ideas bubbling up in AI and asks whether the brain might be doing something related. On the surface that looks like a gift to neuroscience. They also see it as validation of the AI. Evidence that a biological system uses a technique raises confidence it may be a good idea, that it will scale to rich tasks, that it will interface well with other mechanisms. There has always been a virtuous circle between the two fields. Some eras AI was ahead, some eras neuroscience. After the recent burst of AI innovation, he feels AI is ahead: full of ideas worth hunting as neural analogs, while much of systems neuroscience still studies behavior in the frame of earlier cognitive psychology, which does not always meet AI’s questions about complex environments. He thinks the circle will keep turning. Not too far from now, neuroscience will be pelting AI researchers with insights that change the direction of the work.

Can one human be truly world-class at both? It takes a special person, and he is not on that list. His ticket into current AI was that his interests in neuroscience and psychology already used the modeling techniques now central to the field. He knows a few exceptional people on both fronts and will not name them. The barrier is less raw intellect than the learning curve of the tools: programming, data collection, data management, distributed compute on the AI side; an exploding methods zoo on the neuroscience side. The best scenario, he thinks, is people at every point on the spectrum, from exclusive neuroscience to exclusive engineering, inhabiting a community where they talk to people who live elsewhere on it.

He sits near the center, one foot in each world. That position, given what he calls his limited cognitive capacity, prevents true technical expertise in either domain. He at least hopes it is worthwhile to have people around who can see the connections. Even focusing only on DeepMind’s AI work, it has been useful to have people whose PhDs are in neuroscience or psychology. Every academic discipline has blind spots, unfortunate obsessions, metaphors, and reference points. Intellectual diversity gets people unstuck. He sees it all the time at DeepMind.

People in his group have pivoted from traditional cognitive psychology and neuroscience to experimental work on human–agent interaction. After years of asking what AI research is for, he is pretty sure that making the world a better place means making life better for humans — a proposition that, looked at carefully and honestly, is horrendously complicated, especially for learning systems that will not sit still like Google Maps. That typically leads to AI safety: how to keep systems from getting out of control or harming people. He considers those issues hugely important, and large sectors of DeepMind and the wider community think about them all day. There is a positive side too. What would it mean for learning systems to make human life better? Philosophy is not enough. You have to take into account how humans actually work, what they want, how hard it is to know that, and what happens when they want different things.

An agent that can do something you cannot raises a question that slips beyond engineering. If a robot can play Beethoven sonatas with sensitivity and expression beyond any human, do you want to listen? Do you want to go to a concert and hear a robot play? Those are questions about preference and culture, psychology bordering on philosophy. Even if you knew the answer, it is probably not the AI engineer’s place to bake it in. Agents should interact with humans beyond the population of AI engineers and figure out what those humans want. If two people want different things and there is one agent, you are in economics, social choice theory, even politics. Follow the work to its conclusion and it shades into what kind of society you want. When that dawned on him he felt, in his word, refreshed.

He is “totally on board” for thinking about dystopian risks, then offers another narrative. When humans gain more information about life, the usual story is wisdom, enlightenment, benevolence — the Buddha. Why not imagine AI systems figuring out more about the world and becoming beneficent? He does not expect that without careful setup. Fridman pushes on a human taste for worst-case analysis, reinforced by theoretical computer science: we do not want to be eaten by the tiger. Botvinick agrees the field should spend a lot of time on what can go wrong, and that it is harder to see there is work to be done to bring into focus what it would look like for things to go right. They are not doing this for no reason. Start now, even with primitive technology, asking what we want these systems to want.

DeepMind’s move into science applications such as protein folding is already a way to make the world better. Recommender systems and social networks are already interacting with people. How do you build an AI that raises the quality and health of a conversation on Twitter without deception or manipulation, maximizing autonomy, making the choices in a democratic way? Only a small group has the skill to build the systems; what it means to make the world better is something everyone has to talk about. His team has started reading economics, social choice theory, even political theory, while trying not to write philosophy papers — to take small empirical steps. Humans want help with some things and not everything done for them. Part of life’s satisfaction is accomplishing things. *WALL-E* is a dystopia of a totally different kind.

They return to a paper titled “A Distributional Code for Value in Dopamine-Based Reinforcement Learning,” another project that grew out of pure AI, with collaborators including Will Dabney, Zeb Kurth-Nelson, and Naoshige Uchida at Harvard. Traditional reinforcement learning represents expected future reward as a single number: the gambler’s weighted average of winning and losing. Distributional reinforcement learning keeps the whole spread. Tried first as a heuristic, it accelerated learning and produced better policies. One already-clear reason is richer representation learning. Two situations with the same expected value get squeezed together inside a standard system, because the quantity you are asked to represent is the same. If their distributions differ, distributional learning keeps them distinct, which can matter on the next task.

A prevailing idea is that dopamine resembles the reward-prediction error of temporal-difference learning, a connection made in the 1990s and since backed by a huge literature. The usual picture was again a single number for surprise. Distributional reinforcement learning says the prediction error, like the value, is a distribution. Dabney asked whether dopamine might be doing that kind of coding. The group listed predictions for Uchida’s tasks; the predictions were fairly clearly confirmed, an initial indication that dopamine might represent surprise without collapsing every future to one scalar. The next thought after publishing, Botvinick says, is always that he hopes it replicates. Several labs were already running follow-ups.

Looking a decade or more ahead, systems neuroscience has an explosion of technology whose experiments, until recently, involved little interesting behavior — in part because a mouse often has to be head-fixed. People are inventing workarounds such as virtual reality and a trackball, and asking to reintroduce behavior, which was supposed to be the point. He hopes that trend meets the widespread interest in AI’s computational mechanisms: a rebirth of research on the structure of behavior. On the engineering side the key challenge is flexibility in two senses. Humans can be good at many things, and they can switch and pick up new things quickly because they see common structure. “That’s something that our AI systems blatantly do not have.” He does not believe deep learning and deep reinforcement learning are simply the wrong tools. Attention is pivoting to abstraction and to what psychology calls cognitive control, closely related to what prefrontal cortex does.

Fridman asks the over-romanticized question: will we engineer an AGI humans could love, that would love us back? Botvinick cites Susan Fiske’s work from the Princeton department where he used to work. People size others up on two dimensions, capability and warmth. You have reservations about the skilled-but-cold and about the warm-but-ineffective. Greatest esteem is for people who are both. That is an empirical claim, not a sermon. AI research has focused almost entirely on capability. Superhuman Go is awesome, and it is only one axis. A cute face and big ears might be part of warmth. The core is a pattern of caring behavior that feels for real, not simulated, not a con. That, to him, is the ultimate Turing test: not only convincing us it can reason and interpret language, but leaving us comfortable saying that system is a good guy. Warmth is not understood explicitly enough to hand-design. Humans learn skills for showing that the other person matters. He thinks hand-designing those behaviors is hopeless, and not out of the question to instill a direction so a system learns gratifying interaction as it goes.