A Teenager Learns to Drive in Twenty Hours. A Car Still Cannot.

Yann LeCun sits down with Lex Fridman for a second appearance on the podcast, still chief AI scientist at Meta, still a professor at NYU, still happy to argue. The hook is an article he wrote with Ishan Misra, *Self-Supervised Learning: The Dark Matter of Intelligence*. The dark-matter half of the title is the claim that humans and animals are doing a kind of learning that today's machines do not reproduce. Supervised learning needs too many labeled samples. Reinforcement learning needs a ridiculous number of trials. That, he says, is why we do not have self-driving cars.

Most teenagers can learn to drive in about twenty hours of practice. Even with millions of hours of simulated driving, a self-driving car cannot yet do the job properly. The usual reply is that humans bring background knowledge, and LeCun agrees. The open question is how that knowledge was acquired in the first place. Babies in the first months of life learn how the world works mostly by observation, because they can hardly act on it. They are not being trained on a task and they are not being reinforced. They are building world models.

You do not even need to know how a car works. You already have intuitive physics. Drive next to a cliff, turn the wheel toward the drop, and you know nothing good will come of it. You have never had to fall off. A tabular reinforcement learner with no model of the world would have to fall thousands of times before it treated the cliff as a bad idea, thousands more to learn how to avoid it, and millions more to avoid it in every situation it will ever meet. Self-supervised learning is an attempt to give machines that prior without a human standing over every sample.

The trick, LeCun says, is that a self-supervised setting contains far more signal than either supervised learning or reinforcement learning. He reaches for an old analogy of his that someone once called the LeCun cake. Ask how much the machine is asked to predict, and how much feedback it gets, on every trial. The answer changes the thickness of each layer.

In reinforcement learning the universe hands the system a single scalar: you did good, or you did bad. It often waits a long time to say even that. You cannot learn anything very complicated without an enormous number of those sparse pokes. Supervised learning is richer, but not by much. Train on ImageNet's thousand categories and each sample carries a little less than ten bits. That is a few bits per image, not a theory of the world.

Self-supervision, in the ideal that nobody has quite built yet, looks different. Show a machine a stretch of video, stop it, and ask it to predict what happens next. Let time pass, show it what actually happened, and hope it predicts better next time. The feedback is an entire future clip, not a label and not a score. Infants get something like this for free. Eyes open, the world appears; eyes closed, it disappears. Parents walk around, pick things up, open and close doors. None of that is a task. All of it, LeCun says, is the dark matter of intelligence: common sense accumulated with no reward.

Fridman asks whether filling gaps could be enough: continue a sentence, or blank out words; continue a video, or fill in the frames in between. LeCun thinks it is the field's best shot at the moment, even if it is unclear whether the path reaches human-level intelligence or only cat-level intelligence. An intelligent system, on his telling, is always filling in blanks. Sitting across from Fridman, he is inferring the back of a head he cannot see. He does not know which way Fridman will look, but he knows the host will not dematerialize and reappear three meters down the hall.

In natural language, the recipe has already taken over the field. Show a network a sequence of words, remove about ten percent of them, and train a gigantic net to restore the missing ones. Once that pretraining is done, the internal representation can be fed to almost any downstream task. The same idea has been less successful on images, where progress still often leans on hand-designed data augmentation, and it has not yet succeeded on video. Nobody, LeCun says, has gotten a machine to represent the visual world just by watching.

Which is harder, vision or language? The farther you go, he says, the more they look like the same cake. What he wants is a method that makes them look like the same thing, and currently they do not. A later FAIR paper from Menlo Park would show that masking non-overlapping image patches, Transformer-style, can work without the usual augmentations. Video remains the hole. Getting machines to learn from video is, in his view, the largest uncharted territory in the field.

Learning a world model is hard because a prediction is never one outcome. The world may be deterministic or stochastic; either way, it is not fully predictable. Play a short clip and ask what happens next, and there are many plausible continuations. The set grows with the interval you ask the system to cover. The technical problem is how to represent that uncertainty: discrete alternatives, a continuum of possibilities, something in between.

Language has a cheap answer. A net can score every word in a dictionary, perhaps a hundred thousand numbers, and turn the list into a distribution. *The cat is chasing the _____ in the kitchen* admits a mouse, a laser spot, a few other things. Video does not. You are not choosing among frames on a list. You need to represent an infinite number of plausible continuations in a high-dimensional continuous space, and LeCun says we have no good way to do that. Scoring every word already feels like a dumb trick that happens to work. There ought to be a more compressed representation of the distribution. He does not have it.

Language hides another cheat. When several words are missing, the model's distributions over those blanks are essentially independent. In *the _____ is chasing the _____ in the savannah*, the probability of lion versus cheetah does not constrain zebra versus gazelle, even though lions take larger prey than cheetahs. The independence assumption is false. The reason it survives is that nobody knows how to represent distributions over combinatorial sequences of symbols, whose number explodes with length. An abstract latent representation of text would couple those substitutions. It does not exist yet.

Fridman throws the crude objection: filling blanks is just statistics, not intelligence, not deep concepts. LeCun will not enter that argument. Intelligence may simply be statistics of a particular kind. If the complaint is that current systems ignore causality, he agrees they should not: a world model ought to take the agent's own actions as an input, so it can learn what interventions cause what results. You can also watch other agents and infer that they are like you. Mechanistic explanations do not have to be inserted by human designers. Individuals learn them, or evolution learned them. The question he actually cares about is how.

Machines, LeCun says, do not yet have as much common sense as a house cat. Cats have causal models, a superb grip on intuitive physics and on the dynamics of their own bodies, and they do it with about 800 million neurons. Until we can reproduce that, high-level human cognition can wait. The key that might unlock reasoning later is still the world model. In talks he boils the rest of machine learning down to three challenges. First, get machines to learn representations of the world, which is the self-supervised bet. Second, get them to reason in ways compatible with gradient-based learning, because that is what deep learning actually is. Third, get them to learn hierarchical representations of action plans, the way convnets and transformers already learn hierarchical perception. On that third problem he has no idea.

The second challenge has an old name. Model predictive control has been around since the early 1960s; NASA used it for rocket trajectories. You have a predictive model of the system you want to control. Given the state at time *t* and an action, it emits the state at *t* plus delta. If that model is differentiable, you can unroll a hypothesized sequence of actions, score the final state against a goal, and backpropagate through time to find a better sequence. LeCun calls that a form of reasoning. It is basically planning.

The teenager on the cliff is doing the same thing. The dynamical model of the car does not need to be accurate. Run the movie in your head, refuse the turn, take the first step of the better plan, and replan. Control theorists call that receding-horizon model predictive control. In classical optimal control the world model is mostly written by hand and mostly deterministic. Rockets reduce to first principles. Robot arms can be modeled with careful mathematics. Water, trees, branches, people: humans build abstract predictive models of those collective systems without trouble. Machines still cannot. That, LeCun says, is the big challenge of AI for the next decade.

Data augmentation is the old trick of inflating a training set by distorting images without changing what they are: shift, scale, rotate, skew, add noise. It has helped classifiers since the 1990s. It became central again because many self-supervised vision methods depend on it. The architecture underneath is one LeCun worked on in the early 1990s, in parallel with Geoff Hinton: Siamese networks. Two copies of the same net, sharing weights, see two views of the same thing, whether from augmentation, a moving camera, or two photos of one person. Train them so the output vectors match. Invariance is the easy part.

The hard part is collapse. Without a further constraint the net ignores the input and emits a constant vector. LeCun's answer in the early 1990s, with Jane Bromley and colleagues at Bell Labs, is what is now called contrastive learning: show pairs you know are different and push their representations apart. They built it for signature verification, hoping to store an 80-byte code on a credit-card magnetic stripe for NCR. It worked. The banks did not use it. Contrastive learning still has a problem in high dimension: there are too many ways to be different, so you need huge numbers of negative pairs. SimCLR, from Hinton's Google Toronto group, is a recent way to implement that idea. LeCun is no longer a fan.

What excites him now are non-contrastive methods. You show only positive pairs and train the two nets to be informative about each other, so one representation is predictable from the other. Hinton and then-student Sue Becker proposed a version of this in the early 1990s, then the idea sat for decades. LeCun revived it at FAIR with Stéphane Deny, now at Aalto University, in Barlow Twins, then in a follow-up called VICReg, for variance, invariance, and covariance regularization. BYOL from DeepMind sits in the same family. He calls this collection the thing he has been most excited about in machine learning in the last fifteen years, and the best shot at predictive world models that keep what matters and drop what cannot be predicted, such as the exact run of tiles on a floor as a camera pans.

He does not believe a machine can become intelligent from text alone. The amount of world knowledge in language is tiny compared with what we need. Put an object on a table and push the table: it is obvious the object comes along, and he does not think any corpus explains that. A GPT-5000, he jokes, would still miss it. Grounding has to come through a high-throughput channel such as vision. Masking autoencoders for images, without augmentation, are one new path. YouTube may even contain enough raw video to learn how to be a cat, with some selection. Multitask learning, continual learning, even reinforcement learning he treats as shorter-term engineering. Once self-supervised world models work, he expects those problems to shrink.

Asked about Yoshua Bengio's remarks on consciousness, LeCun opens with a shrug: he does not know what consciousness is. A lot of the talk reminds him of a seventeenth- and eighteenth-century puzzle. Once people understood that the lens of the eye throws an inverted image on the retina, they wondered why the world looks right-side up. With modern science the question is faintly ridiculous. He suspects much of the consciousness literature is of that kind, even as he respects colleagues such as David Chalmers at NYU.

His own speculative hypothesis starts from the world model. The prefrontal cortex, on this picture, is the engine that runs it. When you attend to one situation you cannot attend to another, which suggests there is only one such engine, reconfigured for building a wooden box, driving, or playing chess. That is why we do one deliberate thing at a time. Tasks that repeat sink out of that engine into something automatic. A chess grandmaster answers in seconds while LeCun thinks for fifteen minutes; after enough hours of driving you can talk, until the road becomes unpredictable and you go quiet. Pattern recognition has taken over from model-predictive deliberation.

If that is right, consciousness is not a consequence of the power of our minds but of the limitation of our brains. Because we have only one world model, we have to be conscious. An executive overseer has to configure that model for the situation at hand. If we had as many world models as situations, we could run them all at once and would not need that control. The hard problem of feeling, the sense that this is happening to you, he treats as a useful evolutionary side effect: a signal of ownership over the only model you have, because losing it would be very bad. Nativists who want 3D space, object permanence, or V1 edge detectors hardwired get a similar reply. Those things are easy to learn, some of them in minutes of opening your eyes. What is not learned are the intrinsic drives in the basal ganglia, the objective functions that make humans different from cats.

LeCun is comfortable taking humanity down a notch if building intelligent machines is what teaches us how minds work. He compares it to aerodynamics: we understood bird flight better once we built airplanes. The Chinese Room, the idea that a giant lookup table could mimic intelligence without being intelligent, he calls ridiculous. Strong AI, matching humans in every domain where humans are intelligent, gets an unqualified yes. It is not for tomorrow. It will take a long time, longer than Elon Musk and others have claimed, and longer than some people at DeepMind thought five years ago. It is an amplifier of human intelligence, not a riddle about whether the trick is real.

If such a system is driven by intrinsic objectives, can perceive the world, and has a critic that predicts whether a situation will turn out well or badly, he thinks it will have emotions. Fear is what you get when the critic forecasts a bad outcome. Elation is the forecast of a good one. If the machine is also driven to deal with humans, it will be social, and it will have attachment. The sci-fi image of Commander Data's emotion chip, a module you can switch off, he finds ridiculous. Emotion is not an add-on. It is what an autonomous intelligence with a critic is.

Then the legal science fiction starts. If a mind can be backed up and restored, murder changes meaning, and so does skydiving. Robots, built on something like today's computers, would likely have backups. LeCun's sons are Diablo fans and built a game of their own; he likes the analogy of hardcore mode, where death is death. It is possible, he says, that copying some robots would be made illegal, the way cloning a person is not copying a mind but delaying a twin. A household robot that arrives slightly pretrained, lives with you for five years, learns your life, and then wants to work for someone else poses a sharper question: can you erase what it knows about you to protect your privacy? You cannot erase a human. You can only ask them to shut up.

If robots are designed for deep interaction, he thinks they cannot be deleted at will, any more than a person can. Risk is part of the experience. You have to be able to lose the robot friend, and you have to live with the possibility that it will tweet about what a jerk you are. Declare a system sentient, or built for human company, and wiping it starts to look like murder. Backing it up and then wiping the running copy will make for Supreme Court speeches. The useful effect of the whole discussion, Fridman and LeCun agree, is that robots force us to say what we actually value in humans and in other animals.

Facebook AI Research was created almost exactly eight years before this conversation. When LeCun joined, the AI group had about ten engineers and two scientists. He ran it for three and a half years as director: hired the first scientists, set the culture, and explained to leadership what fundamental research was, how it could live inside a company, and why it had to be open. He now calls that stretch an unqualified success. FAIR published top-level research, released tools such as PyTorch, and, directly or indirectly, shaped the systems Meta is built on. Take deep learning out of Facebook's services, he says, and the company would crumble.

After those three and a half years he became chief scientist, dropped day-to-day management, and went back to his own work on self-supervision. FAIR is now run jointly by Joëlle Pineau and Antoine Bordes. FAIR Labs is bottom-up and scientist-driven; FAIR Excel is more organized, for larger projects that need engineering. Both sit inside the larger Meta AI, which itself sits inside Reality Labs, alongside AR, VR, and telepresence. He would keep the FAIR acronym and let the F stand for Fundamental. Brand people were still hesitating. Haptic gloves belong to Reality Labs research; the robot touch-sensing work he had been posting about is a FAIR project.

Fridman asks him to defend Facebook against a press that paints the social network as a net negative. LeCun's answer is blunt. The company described in some media is not the one people know from the inside. He is a vice president. He does not sit on content-moderation decisions, but he does not see the evil being described. Study after independent study, he says, fails to show that Facebook polarizes people or that more time on the platform makes political conflict worse. Researchers paid people in the former Yugoslavia to stay off Facebook around the anniversary of the Srebrenica massacres; those people became more polarized, while heavier Facebook users became less so. Stanford economists found U.S. polarization rising for forty years, before Mark Zuckerberg was born. West Germany and Denmark use Facebook as much and have been getting less polarized.

If you want a scapegoat you can find one. If you want a cause, he says, you cannot pin it on Facebook, and he is angry that deeds done by others, including the owner of the *Wall Street Journal* that ran the coverage, get charged to the company. Fridman offers the printing press. Early printed books included the Bible, which let people read scripture without a priest, which helped produce the Protestant Reformation and two hundred years of religious persecution and war. Social networks, LeCun says, have not been nearly as bad as the printing press, and nobody would call the printing press a bad idea. Zuckerberg, on his account, reads papers, including LeCun's, when he decides to learn a subject. The personality in the press, he says, is just wrong.

At NeurIPS, LeCun posted that a paper he co-authored had been rejected, with *proudly* in scare quotes. The paper is VICReg, a non-contrastive method for joint-embedding architectures. He uses the moment to explain why he abandoned the generative path. One way to handle uncertain video prediction is a latent-variable generator: vary a hidden vector and emit different plausible next frames, which still means predicting pixels. The other way runs the future through a second net and learns representations of the observed clip and the continuation that are maximally informative and mutually predictable. Unpredictable detail is allowed to vanish. A year and a half earlier, in the dark-matter essay with Misra, he was still advocating the first path. He has completely changed his mind.

The first author of VICReg is Adrien Bardes, a resident Ph.D. student at FAIR Paris, co-advised by LeCun and Jean Ponce of École Normale Supérieure and Inria. It follows Barlow Twins, from Stéphane Deny, Li Jing, Jure Zbontar, and others at FAIR. Reviewers said VICReg was not different enough from Barlow Twins. LeCun's view is that it is Barlow Twins with a few bugs fixed, and that this is the version people will actually use. He is used to papers being rejected for a while. Some get rejected and then heavily cited because they work. He cares whether the method is still used in a decade. The conference stamp is secondary.

Computer science is unusual in how much weight it puts on conferences, which at least move ideas into the community faster than journals. The field's exponential growth means most reviewers are extremely junior. Inexperienced reviewers, he says, make their lives easy by hunting for a flaw. Most papers have flaws, including the good ones. What should matter is whether there is a new idea likely to change how other people think, even if the experiments are imperfect. Incremental gains on a fashionable benchmark sail through. Papers that strike at new concepts often do not. Speech recognition had the same disease of chasing numbers.

The escape hatches are arXiv, Twitter as a kind of open review, and systems that treat review as a public recommendation rather than a gate. A decade or more ago, around the founding of ICLR with Yoshua Bengio, LeCun wrote a proposal: post the paper in a repository; any reviewing entity that lists its members may evaluate any paper; there is no exclusive venue; comments are public and signed as a group; entities whose evaluations predict future success gain reputation. A master's student in library science and computer science even worked out formulas. ICLR kept open reviews and, in his view, drifted back toward a conventional conference for everything else. He is saddened that the community is not being innovative enough, and that the hunt for fairness and countable credit is slowing science without delivering fairness.

What pulled LeCun into neural nets was emergence: complex behavior from simple elements in interaction. As a student he found a book on the Chomsky–Piaget debate in which Seymour Papert praised the perceptron, then dug into 1950s and 1960s volumes on self-organizing systems. Many of those papers were by Heinz von Foerster, a German physicist who built the Biological Computer Laboratory at Illinois. Von Foerster's example: in zero gravity, a box of bar magnets, shaken gently, sticks together into a complex structure. That is self-organization. Neural nets, in many respects, are the same mystery. The universe seems to grow more complex even as the second law of thermodynamics says disorder should win. We do not understand it, he says, and neither do physicists or biologists.

We also do not have a clean measure of complexity, which makes talk of things becoming more complex slightly unhinged. Kolmogorov complexity is the length of the shortest program that emits a bit string, but it is defined only up to an arbitrarily large constant that depends on the choice of computer. In a Bayesian frame, complexity is the negative log probability, which looks well defined until you notice that you still need a prior, and the prior plays the same role as the choice of machine. Every measure we have carries an arbitrary additive constant. Without a better handle on the quantity, a theory of how systems become more complex, or of intelligence and evolution, is out of reach.

His demonstration is almost a joke, and then it is not. Take MNIST digits. Neighboring pixels are correlated; the set has structure. Apply one fixed random permutation to every pixel. The images look like noise, more complex, even though their absolute complexity has not changed. Hand someone glasses that undo the permutation and the mess becomes simple again. Aliens who saw the universe through permutation glasses would treat our order as heat and their order as structure. Locality itself might not be shared. That is why he doubts we will recognize alien life, and why he thinks a theory of intelligence will have to wait on a notion of complexity that admits it lives in the eye of the beholder.

LeCun is a tinkerer. He taught himself electronics in high school because a cousin had an analog synthesizer that needed sequencers. He played recorder, crumhorn, and a little oboe in a high-school and early-college orchestra, Baroque and Renaissance music, and always wanted to improvise. The shortcut, short of learning saxophone, was an electronic wind instrument: saxophone-like fingering, synthesizer timbre. He has used Yamaha and Akai EWIs since the late 1980s and has never been satisfied. You hear John Coltrane play two notes and know it is Coltrane. You hear Miles Davis on trumpet and know it is Davis. The sound is physiology, the shape of a vocal tract. Electronic instruments still do not capture that. At Berkeley years ago he met David Wessel, who had founded a music-technology center around the same question. In the pandemic, in a New Jersey workshop that doubled as his Zoom room, he finally started building his own.

The workshop floor holds microcontroller circuits and odd flying machines. Flight is a family disease. His father, a mechanical engineer who taught himself electronics, built radio-control systems in the late 1960s and early 1970s and put model airplanes into LeCun's hands. Before consumer drones, LeCun built his own quadcopters with gyroscopes and accelerometers and wrote the firmware. Once you could buy the thing, he stopped. It was not fun anymore.

His advice to a young person who wants to work on intelligence is to get captured by big questions: what intelligence is, what the universe is made of, what life is, even what time is. Then learn methods with a long shelf life. If the choice is iPhone programming or quantum mechanics, take quantum mechanics. You may never be a physicist, but you will meet path integrals, which are the same formula as Bayesian integration. Learn classical mechanics for Lagrangians. Learn statistical physics, because the mathematics of machine learning was largely built by statistical physicists in the late nineteenth and early twentieth centuries, and more recently by people such as Giorgio Parisi, whose replica method just won a Nobel. If you study electrical engineering, do not skip signal processing: Fourier transforms sit under graph neural nets.

On the applied side he points at AI for science. Colleagues at Meta and FAIR started Open Catalyst, an open collaboration that uses deep learning to design compounds and materials that would split hydrogen from oxygen efficiently. Do that with electricity, cover a desert with solar panels, ship the hydrogen, and he says you have essentially solved climate change. Fusion is another bet: plasmas are unstable, and deep learning might find a controller that holds them. John Platt's group at Google is working on that. Pascal Fua at EPFL trained a convnet on computational fluid dynamics so the drag and lift of a shape become differentiable, and you can backprop the geometry. Twisted bilayer graphene becoming a superconductor at some magic angle is the kind of emergent fact first principles still miss. After all of that, read some literature and some history. The technology has to run in the human world, and the human world is complicated.