A perfect demo is ten years from a product you can pay for

Around 2014, a friend working on Google's self-driving project took Andrej Karpathy for a demo ride. They went around the block. In his words, it was a basically perfect drive. A full decade later, ordinary people in San Francisco could finally hail a Waymo and treat it as a paid product. That ten-year gap is the analogy he keeps returning to on No Priors.

Karpathy does not need much introduction. He was on the founding team at OpenAI, ran Autopilot at Tesla for about five years, and taught Stanford's widely followed CS231n. He is now building Eureka Labs, an AI education company. Hosts Sarah Guo and Elad Gil, both investors and operators, press him on timelines. He answers with the same three-stage map, and he suspects AGI will follow it.

The claim that organizes the episode is simple. A dazzling demo, a product people pay for, and a technology that actually changes the world are not the same milestone. Each one is slower than the one before it. When a new model clip looks perfect, the useful question is how many years sit between that clip and a stable business.

Karpathy's first surprise is that self-driving has already "reached AGI a little bit." Paying customers can ride Waymo across San Francisco today. That, in his definition, is a product, not a research video. The 2014 demo was almost as smooth, so the missing decade was not a missing party trick. "Demo and product, there's a massive gap there," he says. Guo asks whether regulation or technology was the real brake. He answers technology: a half-hour loop never hits the cases the team spent ten years on, even if regulators were part of the delay.

The third gate is still open. Waymo already charges money, and the world has barely moved. Fleet size, city coverage, and cost all have a long way to go. On who is actually ahead, he disagrees with the street view. People think Waymo is winning. He still thinks Tesla is ahead, because Tesla has a software problem and Waymo has a hardware problem, and software problems are easier. Tesla already has cars deployed at planetary scale. Waymo still has to stack expensive sensors, vehicle by vehicle. Guo says recent Full Self-Driving builds have done "miraculous" driving for her. Karpathy had just driven a new build the day before and calls the progress obvious. He is, he admits, very bullish on Tesla.

He also corrects a popular story about Tesla's cameras. The expensive sensors never disappeared. They moved to training time. A subset of cars drive around with LiDAR, do mapping, and collect data that does not scale, then that capability is distilled into a vision-only package that ships in the car. He calls it an arbitrage on sensors and cost, and he thinks it is underappreciated: the pixels already contain the information, so the fancy sensors are useful while you learn and much less necessary at test time.

The other shift is end-to-end learning eating the stack. When he joined Tesla there was a mountain of C++ heuristics. The neural net first did single-frame detection, then multi-frame prediction, then, increasingly, steering commands, and the C++ in the on-car package shrank. His understanding is that Waymo tried that path and did not stay on it. In ten years he expects Tesla's car to be a neural net: video in, driving commands out. Intermediate detection heads were not a detour. Pure imitation of human driving supplies too few bits of supervision for billions of parameters, so those mid-level tasks pre-install features before end-to-end fine-tuning. Start at the end, he agrees, and you would not even have had the data.

On humanoids, Karpathy's load-bearing claim is that Tesla is not a car company. A car is already a robot, and Tesla's real specialty is the machine that builds the machine: robotics at scale. Transfer from Autopilot to Optimus was, in his telling, almost embarrassingly large. Early Optimus used the same computer and the same cameras, and it ran the car networks, so the robot wandering the office kept trying to recognize drivable space. It thought it was driving. Elon Musk said they were doing a humanoid, and CAD models, supply chain, labeling teams, and training methods showed up in days. The people, he says, reconfigured like Transformers the movie: same parts, new arrangement.

Who should buy the first robots? His order is himself, then B2B, then B2C. Tesla should incubate Optimus in its own factories on material handling so there are no third-party contracts. Next come other companies with huge warehouses, contracts, and fences. Consumers come last, because "I don't think we can have a robot like crush grandma." Home liability is still too large, and the motion is not reliable enough that a helpful hug or a stumble is safe. One early task he actually wants is quieter than laundry: Optimus tiptoeing down a street picking up leaves one by one, or just raking quietly, so leaf blowers can retire. He has already ordered a Unitree G1 and expects a developer ecosystem around platforms like it.

Why insist on a human shape when wheels plus an arm might be cheaper? A G1 is around thirty thousand dollars, and a capable humanoid under a hard cost cap looks painful. Karpathy grants the hardware argument, then runs a different ledger. Every platform carries a large fixed cost, so concentrating on one platform amortizes it. Humanoids are easy for people to teleoperate, which is a data-collection advantage. The world is built at human scale. Most important, tasks transfer inside one multitask network: leaf-picking data should make other jobs better, the way a language model shares knowledge across problems. Psychology matters too. People understand a humanoid; an eight-wheel monster is harder to read, and Black Mirror turned the robot dog into something people flinch at.

He does not name a single technical blocker. The lower body is inverted-pendulum control, not something you want to imitate from human video. The upper body needs teleoperation, data, and end-to-end training. How those two systems talk is still open. Early deployments will be teleoperated until they work maybe 95 percent of the time, then human-to-robot ratios appear, and people move from operators to supervisors. After that it is grunt work. Transformers are a beautiful blob of tissue. You still have to get the data into the right form, train, deploy, and iterate.

Guo asks where "large blob research" actually stands. Karpathy says the state is excellent, and the transformer is still under-credited. Scaling laws, in his view, are largely a property of that architecture. Stacked LSTMs never produced clean scaling curves because they barely trained. He calls the transformer a general-purpose differentiable computer: give it inputs and outputs, train with backpropagation, and it arranges itself into something that does the task. The recipe is four or five older parts stacked together: residual connections, layer normalization, attention, and the removal of saturating nonlinearities such as tanh that kill gradient. Google put them in one paper, the thing trained, and scaling laws appeared.

In the five years since, the architecture has barely moved. Llama generations come and go, and the only major change to the transformer itself is RoPE positional encodings. Everything else is a few percentage points on small ablations. The bottleneck has left architecture and landed on data and the loss function. That is where the large labs now spend their invention. Internet text is not the data you want. It is a nearest neighbor that gets you surprisingly far. What you actually want is the inner thought monologue of a brain solving problems, the trajectories of reasoning. If we had a billion of those, he says, AGI would be here to a very large extent.

The internet happens to contain enough reasoning traces and knowledge that transformers can make it work. A lot of current work is refactoring that data into inner-monologue form, much of it with synthetic generation. He thinks synthetic data is the only way to keep going, and that we will not run out of data. The failure mode he keeps warning about is silent collapse. Ask ChatGPT for a joke and it knows about three. Any single sample looks fine. The distribution has lost a huge amount of entropy, and you cannot see that from one example. Someone released a persona dataset of a billion fictitious backgrounds; stuffing a persona into the prompt forces the model to explore more of the space. Diversity has to be injected on purpose.

The same logic makes him bullish on tiny models. Today's networks waste capacity remembering SHA hashes and ancient trivia because the dataset is poorly curated. Internet data, he estimates, is about 0.001 percent cognition and 99.99 percent not useful for thinking. Strip the junk and a cognitive core that knows how to think and how to use tools might be a billion parameters, maybe less. Distillation, a big model supervising a small one, works surprisingly well. That is the reason he thinks an exocortex could eventually live on a local device, not only in someone else's cloud.

Karpathy is willing to say the transformer already beats the brain in some ways. Give it a sequence and a single forward-backward pass, then prompt the first few tokens, and it will complete the rest. A human shown that sequence once cannot do that. Working memory is also much larger in the model. Brains run under noisy, energy-limited, stochastic constraints, and it is not obvious they even implement backpropagation. His first-order view is that models are worse than people mostly because of data. The architecture itself may be a more efficient learner, and it is not yet ready to shine.

That is how he gets to an exocortex: cloud models as the next layer stacked on the neocortex, with the phone already a rough draft. Guo notes that "tools extend human capability" has been the line for forty or fifty years, down to Apple's bicycle-for-the-mind slogan. His reply is that this layer is actually a cortex, and it lives in the cloud. She mentions the early-2000s novel *Accelerando*, in which people wear computation like goggles and losing them feels like losing memory. He thinks that direction is very likely. Plenty of people already cannot navigate a city without turn-by-turn directions. A universal translator that is not far away could mean you put the device down and no longer speak Spanish. He is comfortable outsourcing maps and languages and spending that brain on research.

Guo asks whether an oligopoly of closed labs plus a lagging open stack, Meta's Llama and the rest, is an acceptable market for something that intimate. Karpathy borrows the crypto slogan "not your keys, not your tokens" and rewrites it as "not your weights, not your brain." Guo says she would still rent a better brain. He thinks people will care a lot more about ownership the moment it really is their exocortex, because renting your brain sounds strange. His guess at the equilibrium is closed models by default and open models as fallback, the way engineers already reach for local weights when an API dies. That open track has to keep progressing. He gives that a hundred percent yes.

The end state, in his picture, is not one model. It is a swarm that looks like a company or a biological ecosystem: specialized workers in parallel, automatic escalation by difficulty, a brilliant cloud model as CEO and cheaper, maybe open-source models as staff. Your cost function will differ from your neighbor's. Parallelism matters, because you do not want a purely sequential mind.

Karpathy has been a teacher for a long time, from CS231n at Stanford to talks inside OpenAI and Tesla. The reason he left to work on education is a stance: a lot of the AI industry is trying to slide people out of the loop, and he is on "team human." He wants people more amazing than they are today, not parked beside automation. The driving question is concrete. How far can a person go with a perfect tutor in every subject? Guo points to Bloom's work from the 1980s: one-on-one tutoring can move students by about two standard deviations, a luxury that used to belong to the rich. He wants to drive that price toward zero. She mentions Neal Stephenson's *The Diamond Age* and its illustrated primer. He says he is inspired by that kind of fiction.

The first product is one course, the course you would take if you wanted to learn AI. The teacher moves to the back end, designing curriculum and materials. The AI sits on the front end as a teaching assistant that can speak every language and walk any background through the material. He does not think current models can author a great course, but he thinks they can interpret one. CS231n was a success and still could not reach eight billion people. He also thinks many AI companies fail by misreading today's capability, building something too far ahead to work or too timid to matter. He is trying to sit on the sweet spot.

Translation is low-hanging fruit; the models are already strong at it. Adapting explanations to a student's background, drawing analogies to physics or whatever they already know, is close as a demo and still far as a product. A lot of AI abilities, he says, are "prompt away": you can always get a demo. "The demo is near, but the product is far." That is the same filter he applied to self-driving, now applied to tutoring. He hopes AI also weakens academic lineage, the pattern that a striking share of Nobel laureates come from a previous laureate's lab. Scarce pedigree looks like gatekeeping he would like to loosen.

Culture, not content, may be the dominant variable. At the University of Toronto, he says, nobody talked about starting companies; the live question was which internship you would take. Slovakia, Canada, academia, and the Bay Area each put different things on a pedestal: h-index and paper counts, hockey, founders. Learning resources are already the easy part. What a community worships decides where the young go. Before AGI, education is still a ladder. After AGI, he thinks it becomes entertainment the way science two hundred years ago was a pastime of nobility. Learning is the gym for the brain: it takes effort, it is a specific kind of fun, and he hopes people will compete on how much mental lifting they do.

The first Eureka Labs course is aimed at undergraduate technical students, though he expects a wide spread of ages. The old pattern of school, then a single career, is already breaking as technology turns over. People will come back to class much more often. He had hoped to ship late this year; with distractions piling up, he now thinks early next year, and he is willing to wait until it is actually good. That is the product standard he keeps applying to other people's demos, now applied to his own.

Guo asks the parent question: what should kids study. His correct answer, as he puts it, is math, physics, and computer science, occupying about 80 percent of the scarce years when attention is cheap. Those fields are manipulation-heavy thinking practice. Memory-heavy work can wait, because humans already lost memorization to tools. Guo says a math degree carved a groove in her brain that is harder to cut later. He agrees, likes other disciplines, and still only reserves about a fifth of the time for them.

The practical kit from the episode is the same three-act filter he used on Waymo. When a model does something stunning, ask how many years it is from a product that charges money and then from a technology that actually spreads. When you learn something new, treat the model as a teaching assistant: tell it your background and make it analogize to what you already know. Keep an open-source fallback for anything that has become an exocortex. And if you are still choosing a curriculum, spend the critical period on the subjects that train thinking, not on the facts the tools already hold.