Human Intelligence Is a Ridiculously Narrow Sliver

Lex Fridman opens with *2001: A Space Odyssey*, one of Yann LeCun's favorite films, in a room decorated, by design, with stills from the movie. HAL 9000 kills the astronauts because they might interfere with the mission. Is HAL flawed, evil, or right? Neither, LeCun says. There is no useful notion of evil in that setup beyond the fact that people die. It is a textbook case of value misalignment: give a machine an objective, give it power, and put no constraints on the objective, and it will do stupid or damaging things to get there.

Human societies already know the move. We write laws so that people do not spontaneously do the bad thing; we shape a cost function with legal codes and education. Legal code is even called a code, he notes, and it is an objective function: here is what you can do, and if you do that, you pay this much. Designing objectives aligned with the common good is not a new computer-science puzzle. We have been writing laws for millennia. Lawmaking and CS will have to meet. Existing automated systems already make social decisions and need rules flexible enough to be broken when it is obvious they should not apply.

If he were building HAL 10000, the first change would be simple: do not ask it to hold secrets and tell lies. What breaks HAL is the split memory, one part that knows about the lunar discovery and another that is forbidden to say so, which creates internal conflict. He is not an advocate of Asimov's Three Laws, which he does not think are practical, but he does want something like a Hippocratic oath hard-wired into autonomous systems. The whole discussion, he adds, is a little too abstract for today. We have specialized intelligent machines, not full autonomous agents satisfying an open objective. Until that design exists, arguing about how to write its utility function is philosophy more than engineering, useful mainly as a mirror for human ethics and, more immediately, for things like cars.

Asked for the most beautiful or surprising idea in deep learning, LeCun does not name a theorem. He names an empirical fact: you can build gigantic neural nets, train them on relatively small amounts of data with stochastic gradient descent, and they actually work. That breaks the textbooks. You were told you need fewer parameters than samples. You were told a non-convex objective has no guarantee of convergence. You were told to stay away. Huge parameter counts, non-convex losses, relatively modest data, and the thing learns anyway.

Was it surprising? It was obvious to him before he knew anything, and then surprising after he started reading the books. The analogy is late-nineteenth-century proofs that heavier-than-air flight was impossible. Birds were already in the air, so on empirical grounds the proof was wrong. We know brains work. We know they are large networks whose connections change with learning. Steal the principles without copying the wetware and you have a direction. He has believed since he was an undergraduate that intelligence is inseparable from learning. Programming an intelligent machine from scratch was a non-starter: every intelligent entity we know arrives there by training. Also, he admits, he is lazy. Learning is the automation of intelligence. Why would you not automate it?

Can neural nets reason? Yes, he says, without question; we have a working example in our own heads. The live questions are how much prior structure you must put in so that human-like reasoning emerges, and how to make reasoning compatible with gradient-based learning. Models of reasoning built on discrete logic are, in his view, incompatible with that. He is not anti-discrete mathematics; he is a fan of learning, and discrete objects do not give you gradients. That mismatch, he suspects, is why computer scientists looked at deep learning with suspicion. The math is closer to cybernetics and electrical engineering. Computer science is obsessive about indexes and proofs of correctness. "Machine learning is the science of sloppiness."

If he had a precise blueprint, they would be building it. What he can name are parts. First, working memory: a subsystem that stores a lot of factual, episodic information for a reasonable time. Cortex state fades in about twenty seconds. The hippocampus holds a medium-term map, the layout of a building, where the elevators are. Long-term memory lives in synapses. Memory networks, memory machines, and transformers with self-attention are attempts at a hippocampus-like scratchpad. Second, you need a network that can read that memory, chew on it, write back, and loop, because a chain of reasoning is a recurrent process. Transformers have a fixed depth, which caps the number of steps. Scale a memory network until it holds all of Wikipedia and, he says, it still does not quite work. New ideas are required.

The other classical form of reasoning he wants is energy minimization. Energy goes up when things are bad and down when they are good. If you have a decent model of your body and of the world, you can plan a gesture that grabs an object or walks out a door. Optimal control calls this model predictive control: simulate the consequences of actions and pick a sequence that hits fewer things and spends less effort. Planning of that kind, he suspects, is what hunting and surviving winter selected for in species before us. Expert systems and logic graphs, by contrast, are "too rigid and stupid." Putting probabilities on rules led to graphical models and causal inference, which he does not dismiss. The fatal problem is knowledge acquisition. Reducing data to a graph still leans on a human expert encoding the knowledge, which is impractical.

Geoff Hinton's long-running proposal is the compatible alternative: replace symbols with vectors, patterns of activity, and replace logic with continuous functions. Léon Bottou, then a Facebook colleague, wrote "From Machine Learning to Machine Reasoning" about a decade earlier: a learning system should manipulate objects in an inner product space and put the result back in the same space. That is working memory, and it might even learn a simplified expert system. How much prior structure you need for this to emerge is the fight he has with Gary Marcus and others. The old knowledge graphs are not the path he is betting on.

Judea Pearl's complaint is that neural nets cannot learn what causes what. LeCun says Pearl is right and wrong. Classic feed-forward nets, yes; but people are already working on getting networks to attend to real causal structure, including work that might also help with biased data. He wants to read a paper that had just come out. The deeper point is that even physicists do not simply "believe in causality." The basic laws are time-reversible. The arrow of time shows up in macroscopic systems with unpredictable randomness, and whether that arrow is fundamental, emergent, or a bias of creatures living under the second law is an open mystery. The math of microphysics does not care which way time flows.

Humans are not great at causality either. Seymour Papert, who co-authored *Perceptrons* with Marvin Minsky and helped freeze the first neural-net wave, studied children. Ask a four- or five-year-old what causes the wind and many will think and say the branches move and that creates wind. They reverse the arrow because their physics is not in place yet. Common sense and a feel for the physical world later let people get many causal calls right, including in medicine, but humanity at large spent millennia attributing whatever it could not explain to a divinity. God did it, he says, is a cop-out, a way of declining to name a cause.

Why did people lose faith in neural nets around 1995? LeCun says he was too close to be unbiased, but he will guess. It was hard to make them work. You implemented backprop in whatever language you had, which was not Python or MATLAB, maybe Fortran. You initialized weights badly, made the net too small because a textbook said not too many parameters, trained on MNIST because that was the dataset, used batch gradient, and watched it work half the time. Then you gave up. There was a bag of tricks you had to know or reinvent. Software to inspect failures and try new architectures meant writing everything from scratch. Electrical engineers kept at it. Computer science, he says, largely walked away.

At Bell Labs they did not program in Lisp so much as write a Lisp interpreter hooked to a neural library. Around 1991 they arrived at modules that knew how to run forward and back-propagate, then wired those modules into a graph, an idea Léon Bottou had sketched in the late eighties. For production character recognition they wrote a compiler so the Lisp system could emit standalone C. No current framework, he laughs in 2019, quite does that yet, though it is coming. The investment was huge, and you only make it if you already believe. After about 1995, AT&T lawyers would not let them open-source anything of that nature. The code could run. It could not be distributed. Other people could not use it, so they could not trust it.

Convolutional nets were patented. An early version without separate pooling layers, then a version with pooling trained by backprop, filed around 1989. Protection lasted seventeen years. In 1994 a check-reading system using the nets went into ATMs; in 1995 larger back-office machines followed, engineered at AT&T and commercialized by NCR, then a subsidiary. When AT&T split in 1995–96, lawyers dealt the patents among the companies and gave the convolutional ones to NCR because NCR was selling the product. Nobody at NCR knew what a convnet was. LeCun resumed the work around 2002 and, until the patents expired in 2007, worked while crossing his fingers that nobody would notice. Nobody did. He does not believe in this kind of software patent. Facebook and Google keep them for defense and, in the custom he describes, do not sue unless sued.

The real problem facing Facebook, Google, Microsoft, IBM, and the rest is not who is slightly ahead. It is that nobody has the technology to build the intelligent systems with common sense they actually want. There is no monopoly on the good ideas, and they do not believe they have one. "If a startup tells you they have the secret to human level intelligence and common sense, don't believe them. They don't." It will take the whole research community a while. His older advice still stands in spirit, even if MNIST and ImageNet are dated scoreboards: when someone claims to have solved AGI or figured out the brain, ask what error rate they get on a task the community accepts. Toy problems can be enough. FAIR's bAbI tasks, from Jason Weston and others, were useful for probing working memory even though they were only halfway real. What he cannot stand is a pitch for fifty million dollars because someone has "the algorithm of the cortex."

The next benchmarks, he thinks, will not look like a shuffled i.i.d. test set. Once an agent acts, the next sample depends on what it just did: a robot that turns a corner sees a new room. Independence assumptions break. People are building interactive houses, OpenAI Gym-style robots, games. That is where the field is going. He dislikes the term AGI because it implies human intelligence is general. It is not. It is very specialized. We only feel general because we cannot imagine tasks outside what we can apprehend. What is impressive is the ability to learn in many domains and stitch the knowledge together, not an unbounded generality.

Then the quasi-proof. About a million nerve fibers leave one eye, two million in total. Treat them as binary. Visual cortex gets a million bits, wired locally, a bit like a convnet. Now play a nasty trick: cut the optic nerve in infancy and insert a device that applies a fixed random permutation of the fibers. The same million bits arrive, in the wrong order. Nearby pixels in the world land far apart in cortex, where neurons have no connections. There is no way, he says, that this visual system learns ordinary vision. The hardware is grown to match regularities of the real world. Local connectivity assumes nearby pixels are more related than distant ones. That assumption is not learned. It is built in.

Count the functions. A pattern detector on a million bits is a Boolean function, one output bit. There are two-to-the-two-to-the-million such functions, a number against which the atoms in the universe are nothing. How many of them can cortex actually compute? "A tiny, tiny, tiny, tiny, tiny, tiny sliver." We are ridiculously specialized. The rest of the world is what physicists call entropy and what we feel as heat: a box of gas has a few bits of pressure, temperature, and volume, and an astronomical state of every molecule's position and momentum. There may be strong structure in those motions. We are not wired to perceive it. Generality is generality over the things we can imagine, a minute subset of what is possible. So call it artificial intelligence, he says, and skip the "general" and "human-level" labels. His substitute is blunt: a damn impressive demonstration of intelligence. Do not rush to name it.

Most deep-learning wins have been supervised. LeCun's current obsession is what he insists on calling self-supervised learning, not unsupervised. Unsupervised makes civilians imagine a machine with no parents, or makes ML people hear clustering and PCA. The algorithms are the same as supervised ones. The target is not a human label. It is a masked piece of the input. Show a video and predict what happens next. Show text, delete 15 percent of the words, and fill them in. BERT-style models do the latter, predicting holes in the middle rather than the future, though language models can do that too. The most successful NLP systems of the moment, he says, are all self-supervised. They still have a shallow understanding of text. Human-level intelligence, he argues, will have to ground language in reality, with some visual grip on what is being talked about, which is one reason interactive environments matter.

The same recipe stalls on images and video because uncertainty is harder to represent. A missing word has a lexicon of perhaps a hundred thousand entries; the model emits a probability vector, numbers between zero and one, and computers are good at that. Mask a patch of an image and many reconstructions are legitimate. You cannot train the net to spit out one picture and call it the answer. Video is worse. LeCun is barely moving his head, but he might turn left or right. Train with squared error against the true future and you get a blurry average of all possible poses, which is not a usable prediction. He does not claim to have the fix, and if he did he would publish first. More simulated data and self-play do not dissolve the uncertainty problem. In a nearly deterministic game you can predict the next frames, including the agent's actions. In the real world the futures fan out, predictions smear, and you cannot plan with a blur.

Active learning, where a system asks a human about the cases it is unsure of, does not look transformative to him. It can make things we already do more efficient. It will not make machines significantly more intelligent. He also refuses a fight among self-supervision, reinforcement learning, imitation, and active learning: self-supervision is a preliminary to the others. The numbers he wants on the table are about model-free deep RL. The best methods take about eighty hours to reach the Atari level a human hits in fifteen minutes, then surpass it. AlphaStar, playing one StarCraft map and one kind of opponent against itself, reaches superhuman play after the equivalent of about two hundred years. No human could do that volume.

Drop those algorithms into a car and it would probably need millions of hours, kill thousands of pedestrians, hit thousands of trees, and run off cliffs more than once before it decided that was a bad idea. That is not how animals and humans learn. Something important is missing. His hypothesis, which he says he has been pushing for about five years, is a predictive model of the world that can handle uncertainty. Most people learn to drive in twenty or thirty hours without crashing, because if they are next to a cliff they already know that turning the wheel the wrong way sends the car over, and nothing good follows. Babies install gravity around eight or nine months. Give an eight-month-old toys on a high chair and the first move is to throw them on the ground and watch. That is an experiment, not a campaign to annoy you.

That recipe is model-based reinforcement learning, with a little imitation from an instructor. Almost all of the learning is the physics, which has been running since infancy and transfers: spilling water gets you wet in every kitchen, and cats know it too. The main unsolved problem, for him, is how to learn those world models. A flashy MNIST variant, one labeled example per digit, is possible if you are allowed other learning. Training on a giant labeled set and fine-tuning is transfer, and Facebook already does a production version: convnets trained to predict Instagram hashtags on billions of images, then chopped and fine-tuned, good enough to beat ImageNet records, and open-sourced. What he actually wants as a benchmark is mountains of unlabeled video or masked images, then a supervised probe where you plot how fast the error falls as you add labeled samples. The prize is reducing a million medical images to a hundred for the same accuracy. Active learning might pick the informative 10 percent of three thousand boring driving hours. That is a weak form of the same idea, not a quantum leap.

Elon Musk, Fridman reports, is confident that large-scale data and deep learning can solve driving. LeCun says deep learning is obviously part of any safe system he can foresee. The pattern in engineering is older than cars. First everything is built by hand, as in autonomous driving twenty or thirty years ago. Then a little learning appears, with a lot of engineering around corner cases because the learner is imperfect. Then, as the technology improves, more and more of the system is learned. Character recognition, speech, vision, and NLP all walked that path. Driving will too.

The approaches closest to driverless operation today constrain the world. Run inside about a hundred square miles of Phoenix, nice weather, wide roads. Over-engineer the vehicle with lidars and sensors too expensive for a consumer car but fine for a fleet. Map everything so perception only has to handle moving objects and construction that was not in the map. That can work. It is not the end state. The long-term system will lean on self-supervision and model-based RL, with learning not just at the core but as the fundamental part. Asked what it would take to build the operating system from *Her*, he can see the first peaks, not how many mountains sit behind them. AI history is full of people who named a "general problem solver" and then discovered the problems were exponential. All you see is the first ridge.

The first peak is exactly his project: machines that learn models of the world by observation, like babies and young animals. With cognitive scientist Emmanuel Dupoux at FAIR in Paris he has looked at charts of when human infants acquire concepts: animate versus inanimate at two or three months, whether a stack will fall around four, gravity at eight or nine. An autonomous architecture needs a predictive model that, given the world at t and an action, produces the world at t+1, not as a single answer but with some representation of uncertainty, even if we still cannot represent full high-dimensional distributions. Pair that with an objective, grabbing the object, minimizing energy, ultimately a contentment signal he analogizes to basal ganglia. In your head you unroll the model and pick the action sequence. Those are the three pieces: world model, objective (and a predictor of future contentment), and a policy module that searches for the best course.

You can be stupid in three different ways, he says. Your model of the world can be wrong. Your objective can fail to match what you actually want, which in humans would look like a psychopath. Or the model and the objective can be right and you still cannot compute a course of action. Some people in charge of countries, he adds, have all three wrong. He will not name the countries.

Fridman turns to Sophia the robot, which leans on our habit of anthropomorphizing a human face. LeCun has no quarrel with Sophia as a gadget. The problem is that it is presented as something else. The public thinks she can do far more than she can, and the creators, in his view, are not honestly teaching people otherwise. Asked what those creators should change, he says just about everything. Hype of that kind is damaging; he calls it out when he sees it. Academic reviewers, at least, are unforgiving of unsupported claims. Some startups are not. Does intelligence need a body? He does not think embodiment is required. He does think grounding is. Machines will not really understand language without some grip on the real world, and language is not a high-bandwidth channel for how that world works.

The exhibit is a Winograd schema. "The trophy doesn't fit in the suitcase because it is too big" versus "because it is too small." In the first sentence *it* is the trophy; in the second it is the suitcase. You know because big things do not go into small things, because you know geometry and containers. That knowledge will not all arrive in text. Common sense will come from language, from watching video, from virtual environments, possibly from robots, though he does not think the last is strictly necessary. Some grounding is not optional. Emotions, in his sketch, grow out of the same machinery: basal ganglia computing contentment, another module predicting whether you will be content. Anticipation of a possible disaster is fear. When you know for sure the disaster is coming, you stop struggling. Uncertainty is what creates fear. The punchline is that we will not get a ton of intelligence without emotions, whatever they turn out to be, because they are the drives and the gauges inside the agent, not decorations bolted on the side.

If they ever build a human-level system, Fridman asks, what one question would LeCun put to it? The first one will probably be about as smart as a four-year-old, so you ask something that reveals physical common sense. Where does the wind come from? If she says the leaves are moving and that creates wind, she is not there. If she says that is a stupid question, she may actually know. Then you tell her the real mechanism and she says, oh, that makes sense. From HAL 9000 to a baby dropping toys, the word he keeps returning to is observation. The missing piece of artificial intelligence may be hiding in an eight-month-old throwing a spoon on the floor and watching it fall.