You Might Have to Understand the World to Understand the Letter A

Lex Fridman opens with a blunt question: do we need to understand the brain in order to build one? Dileep George, co-founder of Vicarious and formerly of Numenta with Jeff Hawkins and Donna Dubinsky, says yes. He sets that against Henry Markram's Blue Brain Project, which tries to pour more and more biophysical detail from neuroscience experiments into a giant simulation and switch it on. When the simulation fails to do what you hoped, George asks, what do you do next? You add more details. You cannot debug a system whose design intent you do not know.

Unless you have a theory of how the pieces are supposed to fit together and what each one contributes, he says, you cannot build the thing at a functional level. Blue Brain's original idea was detailed Hodgkin-Huxley neurons, wired according to connection statistics from real experiments, then wait and see. Those single-neuron models are not crude. They capture ion channels, receptors, and spike output in the language of electric circuits, and George would put a lone neuron at about 99 percent right. That still does not tell you how to build a brain.

The analogy is a transistor. An accurate transistor model, plus wiring guessed from experiments, is unlikely to yield a microprocessor. You need Boolean logic and gates first, and only then the mapping of gates onto transistors. Fridman recalls a paper that turns neuroscience's whole toolkit on a known microprocessor, as if aliens had arrived to study computers. The first-pass takeaway was that those tools would not recover even Boolean logic. George knows the paper—*Can a neuroscientist understand a microprocessor*—and adds that, following that methodology, even an electrical engineer would not understand microprocessors. Neuroscientists find valuable things. Those findings have to be assembled in a computational framework.

His own loop is to take neuroscience insights, read them as information processing, and build a functional model that actually does the task, not a model that only reproduces a phenomenon. Building the model fills in pieces biology only hints at; you then go back to anatomy and propose new experiments. Connectivity maps of cortical microcircuits are beautiful, he says, and they do not by themselves tell the story of how the system works. Someone has to ask why the wiring is that way.

Vision occupies a huge fraction of cortex, and it is not a feedforward stack. There are far more feedback connections than feedforward ones. That fact collides with the deep-learning habit of treating perception as layer after layer of feature extraction. Experimental neuroscientists, George says, are systematic and hypothesis-driven because the experiments are expensive; they have probed lateral connections, blocked neurons in V1, and asked what those circuits contribute to visual processing.

One study he was impressed by uses a textured triangle on a textured background. When the percept finally forms, you know both the contours and the interior. Which settles first? The system converges on the edges, then fills the surfaces inward. Researchers can time that dynamics and say which layer in V1 encodes edges, which encodes surfaces, and how feedforward and feedback combine into the final percept.

Illusions make the same point. In the Kanizsa triangle, only Pac-Man-like corners are in the stimulus; the visual system hallucinates faint edges. Neurons signal those edges even though their input is blank. For a real contour they fire immediately; for an illusory one they fire later, driven by feedback. George's picture is that the brain is building a model of how objects behave and constantly projecting that model onto the world. What you see is the combination of that projection and the sensory input.

Fridman reaches for "calculating the difference." George pushes back. He would not put it that way. It is more like asking what is the best explanation for the input, given the model of the world. That inference is iterative, feedforward plus feedback, at every level of cortex. Multiple competing hypotheses try to explain the same evidence, and one can explain another away.

George distinguishes two families of machine-learning models. Neural networks are function approximators: layers of nodes doing dot products. Probabilistic graphical models treat each node as a variable—an edge present or not, an object present or not—and encode the relationships among those variables. Once the knowledge is stored, inference is the question of how best to explain some evidence under that model.

Graphical models can represent causality in the world. Your home alarm can be set off by a burglar or by an earthquake. You are at the office, the alarm goes off, and you head home thinking there is a burglar. On the radio you hear there was an earthquake nearby. Belief in the burglar drops, because the alarm has been explained by the quake. Increasing evidence for one cause pulls the same lower-level evidence away from the other. Two competing causes are fighting for the right to explain one observation. The brain, George says, has a similar mechanism, and you should be able to point at a circuit and say the competition happens here.

In a forthcoming paper on cortical microcircuits—still largely a hypothesis—George proposes that a cortical column encodes a concept, in the sense of a binary random variable: is an edge present, is an object present. Connections between columns encode the relationships among those variables. That is where the knowledge lives about how concepts connect to one another.

Inside a column, thousands of neurons in multiple layers implement a rich, repeated motif, and every column also connects through the thalamus. The hypothesis is that neurons inside the column, together with thalamus, perform the actual computation: inference, including explaining away and competition among hypotheses. Experimentalists have already shown, for example, that a high-level neuron can inhibit another column through a complicated loop via the thalamus. They often stop short of placing those results in an overall inference framework.

Fridman asks whether the "concept" has to be human-interpretable. George says no. You will often be able to find some interpretation because the variable is connected to other things, but readability is not the point. The point is that it is useful as an entity in the graphical model. Cortical microcircuits, in this language, are the intricate within-level circuitry—the analog of a much more structured layer than feature detection plus pooling.

George and colleagues synthesized those neuroscience insights into a computational model, the recursive cortical network, first used to break CAPTCHAs and later for robotic picking and tracking. It is a joint vision system: class, segmentation, edges, interiors—not a specialist for one benchmark. Perception should not be treated as a preprocessor for cognition. The two are interconnected, and a perception system that will later connect to concepts, language, and top-down attention has to be designed with that end in mind.

Top-down controllability means you can close your eyes and inspect an object, zoom into the cap of a bottle, imagine what happens if something hits it. The generative model is not trying to paint a photorealistic scene. Ask someone the color of the letter E in the Google logo: they have seen it hundreds of times and still have no idea. The model is not photorealistic, but it can be manipulated—fill a different color, expand the E, imagine actions never performed. Those are the properties a generative model needs if cognition is going to run it.

In RCN, background is modeled separately from foreground, objects from parts, shape from texture. Those factorizations, George says, can be derived from more general properties of natural signals—the kind of input human vision is tuned for, as opposed to QR codes, which computers classify easily and people do not. Encoding those priors so that inference is fast and distributed is a core problem the vision model tries to solve. Completing the corners of a triangle looks like abstract reasoning; it is a property of the visual system, stored in a form that inference can use.

Compared with a convnet, the feedforward path looks familiar: feature detectors and pooling, stacked. RCN can also run backward, combining the forward pass with generation, which is the analog of cortical feedback. It adds lateral connections among nearby edge nodes to enforce compatibility. Without them, transformations in different parts of the field are uncoordinated, and generated contours come out jagged. The whole thing is still layered. It is not differentiable. It is not trained with backpropagation.

CAPTCHAs exist as human-interaction proofs: easy for people, hard for bots. Text CAPTCHAs were still hard for computers around 2014. George argues they remain a good challenge if the goal is human-like perception. Systems have cracked a lot of text CAPTCHAs, he says, but not the way humans do. He can take a five-year-old who has just learned letters and show any newly generated CAPTCHA from any site. The child reads it, with no training examples from that style. Current deep learning, he says, still cannot be trained on whatever you like and then handed a genuinely new CAPTCHA distribution.

Douglas Hofstadter put it in a talk: the central problem in AI is the letter A. If you can reliably detect all its variations, you do not even need to go on to B and C. An A can be two people leaning together holding hands; it can be made of leaves. "You might have to understand everything about this world in order to understand the letter A." That is why CAPTCHA is a good problem. A child brings structural priors about the world, independent of font. To say we have solved CAPTCHA in the human sense, George says, you have to solve the whole problem.

CAPTCHAs crowd characters together. An R next to an N looks locally like an M. Local evidence screams the wrong answer; the only globally consistent explanation is not M. Inference here means placing local evidence in a global context and often concluding something that conflicts with the local cue. A neural net can brute-force the combinations at training time—amortized inference that looks like reasoning on the fly. RCN aims at dynamic inference: unseen combinations solved from the model's constraints. The system returns a full explanation of the scene, not just a class—which pixels support the A, which contours, what is in front of what. Its errors, George says, look somewhat like human errors.

Why not ImageNet? Those benchmarks fit deep learning's setting: a training set perhaps a hundred thousand times larger than the test set. George wanted the opposite. On versions of MNIST they inverted the usual protocol and reached about 95 percent accuracy with on the order of tens or hundreds of examples, better than other systems at the time. The same model trains quickly, is robust to out-of-distribution perturbations, and they were using it on robotic tasks. Standard benchmarks have momentum, he notes; once a number becomes the number, the academic system pushes people to break it rather than step outside the box.

Fridman puts the awkward question: brain-inspired work is accused of big claims and thin evidence. George stands by the *Science* paper—feedback, recursive inference, lateral connections, joint recognition and segmentation, compatibility with higher cognition and top-down attention. After acceptance, *Science*'s press office wrote that "a new deep learning model solves captchas." The abstract had said the opposite: current deep learning needs lots of data and ignores these insights; this is a graphical model that does inference, not a deep net. The hype machine translated it back into the hottest word of the moment.

Skepticism in science is necessary. What he often sees is a bandwagon effect, especially with newcomers flocking to one approach. He does not treat "brain-inspired" as marketing. He is in the details of biology. It is easy to be misunderstood if people do not see how you actually work. Convolution is an example of when to deviate: cortex is not convolutional. It has local receptive fields but not translation invariance. Shared weights are an engineering trick for data efficiency, and they will matter less once a robot has a moving head and eyes. The fovea and the periphery have very different weights. Understand the principles so you know when to use a trick.

At Numenta with Jeff Hawkins, the high-level hierarchical picture still holds if you blur the details; the details that matter have changed. A decade earlier, George did not want learning algorithms locked to then-known biological mechanisms. New biophysics and ion channels keep showing up. Credit assignment of some kind is required—you have to adjust connections that helped—but it need not be differentiable, and it need not be backpropagation. Expectation-maximization is another family. On the inference and recognition side he is much more willing to be constrained, because you can present a stimulus, time the response, and trace it through layers. You cannot run that kind of experiment on learning, so you do not know whether today's biological constraint is right or premature.

Perception was never for its own sake. They asked what cognitive architecture you need: a perceptual system, a system that learns the dynamics of the world, and something like program learning on top to form concepts. Version 0.1 of that stack is a *Science Robotics* paper on cognitive programs, aimed at manipulation. Imagine meeting someone who does not share your language. You want them to pick up all the red cups from the kitchen counter and put them here. You show a start picture and an end picture. The person has to infer, conceptually, what happened. That is preverbal conceptual understanding: manipulating concepts in the head from images, without words.

From text alone you can extract associations—cows are likely on grass. That is not a concept. A concept is an abstraction of something that happens in the world, grounded so you can simulate it and connect it back. Concepts in the visual and motor systems, learned by interacting with the world, are a prerequisite for any real language understanding. George disagrees with Chomsky completely, including universal grammar. Perception has to be built first even though it will be the last thing fully solved, because a wrong perceptual system will not support a right concept system.

"Sally pounded a nail on the ceiling." Is the nail's middle horizontal or vertical? Vertical. You answered by simulating the scene. Language posed the question; the knowledge is not in the language system. It comes from everyday experience, largely in place by age five, stored in vision and motor systems and accessed through language. Quick associations often suffice and are often right; you can construct cases where they fail and you must actually run the simulation. Their stance is that language is simulation control, and the perceptual-motor system is building the simulation. They built a controllable perceptual system, schema networks for controllable dynamics, then a concept learner that wraps those into programs you can run. Next is grounding simple language in those simulations—the sorts of queries a language model would miss.

George has not evaluated GPT-3 himself. From what he has seen, the 175-billion-parameter model is intriguing: longer coherent generation was already visible in GPT-2, and the weaknesses are visible too. It does not really carry a world state. You get incompatible sentences—"I went up the hill to reach the valley"—or travel that ignores duration. More data will reduce those glitches and stretch coherence. It still will not have a world model it can simulate in order to check whether something is true.

Fridman plays devil's advocate: if you force a system to compress a huge corpus, might abstractions that look like concepts and world models fall out? George says no, because the information is not there. Unless someone has written down, in absurd detail, how everything works—that it is easier to walk forward than backward, that you open a door to leave, that doctors wear underwear—or the model found those facts useful for compressing other text, they are not in the corpus. Text is lower fidelity than interacting with the physical world. Pictures help less than people think; interactivity is the richer signal.

Shannon's book has first-order and second-order Markov models of English; the higher-order samples look more like English. Do they have a world model? At that level, yes. Transformers are more sophisticated and useful. Scaling them is not going to give AGI or natural language understanding. A transformer is still a feedforward architecture, good for statistical text modeling and some video. It does not capture the family of models you need for feedback, causal and counterfactual reasoning, and interventions—actions in the world. George's joke: this is the model that read all of quantum mechanics and relativity, and we are asking it to do text completion. What we actually want is a system that forms hypotheses and revises them from experiments.

Fridman does not want to be dogmatic. Neural nets working at all is surprising, given the nonlinear spaces; finding reasonable local minima is surprising; that reinforcement learning works, and AlphaZero's self-play, are surprising. Maybe the core of intelligence is much dumber than our poetic talk of hierarchical concepts. George grants the possibility. Underneath it is still connections and message passing. He still thinks biology and evolutionary history are clear: we need more structure, world models, and support for dynamic inference.

It is very clear, George says, that you need at least two kinds of memory. One is statistical patterns in the world, the regularities you can compress and generalize. The other is the one timeline of things that happen only once and will not happen again, stored as a stream of experience. The hard part is connecting them: this happened, I should be careful in similar situations, and I need to index that similarity using the world model even though the episode itself is unique.

The hypothesis is that hippocampus stores a sequence of pointers over time. To reconstitute a memory—was it good or bad, should I meet this again—you need cortex to replay it. Retrieval is bidirectional. In a new situation cortex is doing inference; hippocampus, connected to many cortical areas, can give you déjà vu, an index into when this happened. You can then let hippocampus drive cortex, rewinding the experience instead of being driven by current stimuli. Putting that replay back into cortex changes what you see next. Graph neural networks are already a hybrid of neural nets and graphical models; he expects more of that. The direction, from biology, is more structure, world modeling, and dynamic inference.

Brain-computer interfaces sit at the intersection of George's two loves. A BCI talk at Stanford is what first got him thinking about brains. Set aside sci-fi telepathy: intermediate milestones already matter—controlling an external limb from cortical signals, writing into the brain, prosthetics for people who have lost limbs, help for quadriplegics. Even if you magically solve the safety problem of putting millions of electrodes in, you still do not know how the brain will respond. Plasticity is high. Both the machine and the brain will adapt. The hallucinations from such a hookup could be intense.

The more promising path, he thinks, is not decoding the brain's native protocol. Attach electrodes to a patch of cortex that is not damaged and not already used for something else. If it is done from birth, the brain can train that tissue as a high-bandwidth channel. Design the external system to be adaptable, the way mice and keyboards are designed for humans. Two systems adapting, with the main direction being the brain adapting to the machine. Connecting that channel to something like a Twitter stream is a warning, not a product pitch: the intensity is easy to imagine. Surgery and long-term biology remain the hard barrier.

Consciousness does not inform day-to-day engineering, but the company name Vicarious is tied to it. First, you interact with the world, then you can run those processes in your mind without acting—experiencing vicariously. Second, the same apparatus can model another person; you put yourself in their shoes. Third, you can turn that modeling on your own modeling apparatus. That, George thinks, is what gives rise to consciousness. Fridman distinguishes self-awareness from the hard problem: it feeling like something to be that entity, which brings in the possibility of suffering. Mortality, he suggests, is a deep human motivator, a machine that generates goals because the mystery of the end is terrifying.

George agrees mortality is part of human cognition, but sees no reason it must apply to an artificial system you can copy. You cannot clone a person such that the episodic memory stored in their brain comes along. That is not true of an AI. Fridman wonders whether a copyable system is therefore not yet AGI—every known intelligent being has a deadline. George is not too worried about AI developing a motive to kill all humans. Why do that? Just wait. An immortal, copyable system would not inherit the urgency that finiteness creates in us. Murder, as Fridman puts it, seems like a lot of work.

Three books. Judea Pearl's *Probabilistic Reasoning in Intelligent Systems* is deeply technical, but Pearl sprinkles philosophical observations and connects graphical models to how the brain thinks, to attention, and to resources. Causality as a full program came later; *The Book of Why* is more enjoyable. Second is a collection George thinks is called *The Mind's I*, by Douglas Hofstadter and Daniel Dennett. Third is not technical: *The Bishop's Boys*, on the Wright brothers. Flight was treated as unsolvable. People thought the problem was powerful lightweight engines, or how far you could throw the machine from a catapult. After the invention, many still did not believe it.

People say we did not learn from birds: airplanes do not flap. The irony, George says, is that "you don't need to flap to fly" is something the Wrights found by observing birds. Their notebooks have detailed notes on buzzards soaring over thermals. Flapping is not the important part. Propulsion is not the core problem. Solve control, and propulsion will fall into place. That is how he thinks about brain-inspired AI: read off what is essential and what engineering can bypass. Copying details was never the point. Deciding what to copy is a judgment you can only make by actually understanding the brain.

Advice, with a pinch of salt. If your goal is to understand the brain in order to build one, experimental neuroscience may not be the path; computer science, electrical engineering, machine learning, and AI are better, and you can study neuroscience on your own. If you are drawn to discovering something about the brain itself, be an experimentalist. A Redwood Neuroscience Institute panel about ten years earlier, organized by Jeff Hawkins, was asked what undergraduate major to take to understand the brain. The majority said electrical engineering: circuits, microprocessors, information theory, signal processing, continuous math. The downside: you are more likely to be forced to use MATLAB. Everyone should learn to program. It puts data at your fingertips so you can organize it, search it, and, when the science calls for it, build systems.

The meaning of life, George says, is whatever you can stick into it. It is wide open. The perception system is there to understand the environment—to not die, not fall over. Understanding the machinery of the world is what we should be striving for. Then you can push that machinery toward whatever else you want. The rest is whatever is culturally popular. That closing line matches the rest of the conversation. Making AI more human-like is not the objective function. Figuring out how the machine of intelligence works is.