The Cheaper Intelligence Gets, the More of It We'll Buy

Jacob Effron introduces David Luan as the head of Amazon's AGI lab, previously co-founder and CEO of Adept—a company that raised more than $400 million to build AI agents—and VP of engineering at OpenAI through a stretch of its defining breakthroughs. The episode of Unsupervised Learning is a reunion as much as an interview. The two have been friends for more than ten years, long enough for Effron to remember thinking Luan's move to OpenAI looked interesting and, in hindsight, characteristically early.

Luan traces the through-line to robotics. The binding constraint was always how smart the underlying algorithms were, so he started working in AI and has watched the stack start working in their lifetimes. What Effron wants to hit is topical on purpose: the DeepSeek reaction, how far test-time compute goes, what it will take to make agents reliable, and the early OpenAI culture that still shapes how Luan hires and builds.

Effron asks what the broader discourse got right and wrong after DeepSeek, the NVIDIA stock crash, and the claim that the release was bad for OpenAI and Anthropic. Luan still remembers the morning. He woke up to five missed calls, and the last time his phone had looked like that was when Silicon Valley Bank collapsed and investors were ringing to pull money out of SVB and First Republic. He assumed something terrible had happened, checked the news, and found stocks crashing because DeepSeek R1 was out. His instant reaction: people had really missed the memo on what actually happened.

The work itself, he says, was incredibly good, with a team culture and composition he could talk about for a long time. It also sits inside a longer story arc. First the field figures out how to make new ML systems smarter; then it figures out how to make them more efficient. DeepSeek was the tock to the tick of o1. What everybody got wrong was the demand curve. Just because you can produce more intelligence at a lower price does not mean you stop consuming it. If anything, you consume even more. Once the market woke up to that, he says, we were back to sanity.

Effron notes that the base DeepSeek model can be induced to say it is ChatGPT, and asks whether OpenAI and Anthropic will stop releasing models so publicly. Luan does not litigate provenance. He predicts the industrial pattern: people want the smartest models possible, those models are not always inference-efficient, so labs will train humongous teacher models on as much compute as they can get and then render them down internally into something that runs fast for customers. He pictures AI use cases as concentric circles of complexity. The innermost ring is an ordinary chat with a base LLM, something GPT-2 could already do competently. Outer rings are mental math, coding, agents, drug discovery. Every previous ring, he says, becomes almost so cheap as to be commoditized.

Effron points at the test-time compute wave in coding and math and asks how far the paradigm goes. Luan has been leaving a paper trail on the AGI recipe since before it was fashionable. Even in 2020, with GPT-2 out and GPT-3 cooking, the live debate was whether next-token prediction was enough to solve all of AGI. His answer, and the answer of a few people around him, was no. A model trained only to predict the next token is, by definition, penalized for discovering knowledge that was not in the training set. To discover new knowledge you need the other paradigms that have already been shown to do it: reinforcement learning and search. AlphaGo was the moment that went into public consciousness.

The question was always when the field would combine LLMs with RL—systems that hold what humanity already knows and can build on it. DeepMind's early pure-RL path did not work, in his telling, because those models were initialized randomly. Something that knows nothing about the world would take forever to rediscover human language, coordination, and how to file your taxes. That philosophy, he says, has been borne out by how well models that combine both paradigms now work. On domains that are not easily verifiable, such as healthcare or law, he has a strong view: these models are better at generalizing than you think. People who tried o1, noticed a wait, and decided it was a bit worse at chat are looking at "blips on the way to glory." There are already signs that getting better on problems you can explicitly check transfers to slightly fuzzier problems in a similar domain.

The deeper arbitrage is inside a single set of weights. Models are better at determining whether they did a good job than they are at generating the right answer on the first try. All of this RL, he says, is exploiting that gap: force the model to try over and over until it satisfies its own sense of quality. Yann LeCun's recurring critique is that LLMs will struggle with original thinking. Luan's reply is that we already have counterexamples. AlphaGo was original thinking. Early OpenAI work used RL to play Flash games on sites like MiniClip, and the agents learned speedrun tricks—glitching through walls on platformer levels—that humans had never done. Nobody taught those actions.

Asked what research problems actually have to be solved to ship a model like this, Luan starts with organization. If you run a modern AI lab, your job is not to build models. Your job is to build a factory that reliably turns out models, and that framing totally changes what you invest in. Until there was repeatability, he did not see much forward momentum. The last couple of years, in his view, were the arc from alchemy to industrialization, and without that substrate the rest does not work. People gravitate to algorithms because they seem cool and sexy. Looking at what actually drove a lot of progress, he keeps coming back to engineering: giant clusters you can keep up long enough, and a node failure that does not waste the job.

Reinforcement learning raises the engineering difficulty another notch. He describes a coming world with lots of data centers, each doing a great deal of inference on the base model, testing it on new environments a customer has brought, and sending that new knowledge back to a centralized place so the model can get smarter. That distributed machine is full of hard problems. The through-line is unglamorous on purpose. You have to go slow to go fast, and the factory has to exist before the next scientific trick has anywhere to live.

Luan remains super excited about agents, and he dates the itch to 2020 and 2021, when powerful LLMs could write a rap, land a roast, and do three-digit addition acceptably—then fail the moment you asked them to order a pizza. At Google he worked on tool use: exposing affordances so a model can decide when to go do something. The RL literature had always said "agent," the public did not have a word yet, so his group tried "large action model." The world chose "agent," and now everything is an agent and the word means nothing, which he finds very sad. Adept, in his telling, was the first modern agent company, founded in a brutal year to start one. The best open-source LLMs were not good, multimodal image-input models did not exist, and GPT-4V arrived much later. They had to train everything end to end, which he likens to starting an internet company in 2000 and calling TSMC to build your own chip.

The early lesson was that LLMs without today's RL are behavioral cloners. They do what they have seen, and the moment they land in a situation they have never seen, generalization is bad and the model does something unpredictable. Adept was chasing useful intelligence, not a demo that would go viral on Twitter: take the grunt work off knowledge workers who care about reliability. Invoice processing was the hello world. The failure mode is vivid. If the tool deletes a third of your QuickBooks entries one time in seven, you will never use it again. Two years on, reliability is still the issue. He finds OpenAI's Operator super impressive and a cut above Claude's computer use, but both have focused on end-to-end task performance with very low end-to-end reliability and lots of interventions. Real value, he says, is when businesses can fire and forget.

Turning a base multimodal model into a large action model is two jobs. The engineering job is exposing, in a model-legible way, what it can do: APIs, UI elements, a little about how Expedia or SAP works. The research job is teaching it to plan, reason, replan, follow instructions, and later infer what the user actually meant. That is a different problem from ordinary LLM work. Even long math reasoning is still one step with an answer at the end. Agents are multi-step decision-making with backtracking and a need to predict the consequences of an action, including that the delete button is probably dangerous, then learn in sandboxes. He borrows Andrej Karpathy's textbook analogy: pre-training is the exposition, supervised fine-tuning is the sample problems, and RL is the open-ended exercises whose answers are in the back of the book.

If he could ding the field for one thing, it is a massive lack of creativity in how people interface with increasingly smart models. Today's AI interfaces, he says, feel like the early App Store: a button that makes a burp, a beer you pour by tilting the phone. Chat is a low-bandwidth way to get things done. He does not want a seven-turn conversation to decide pizza toppings. Part of the reason is that great product designers do not yet deeply understand the models' limitations, while the people advancing the technology still think of themselves as delivering a black box rather than an end-to-end experience. What he wants is an agent that synthesizes a multimodal interface to elicit what it needs, with shared context—working on the computer together, looking at the same screen, parallel rather than perpendicular.

When do we actually get reliable agents? Operator is impressive, Luan says, but the field is still missing the last chunk, and the analogy is self-driving. Effron cites a 1995 cross-country demo that ran something like 99 percent autonomously; the industry then spent decades on the remainder. Luan does not think agents will wait thirty years, because this time the right tools are in the toolbox. The milestone he is watching is specific. During training, at a major lab, he wants a recipe where he can hand the agent any task, come back days later, and find it 100 percent at it—not humans squeezing out another five points of reliability, but the agent having figured out how to solve the thing.

Could a startup still win if it started Adept today, or will only frontier labs and hyperscalers move the ball? He has a huge amount of uncertainty and a current view. He thinks AGI is not super far away, defined two ways: a model that can do anything useful a human does on a computer, or a model that can learn to do that thing as fast as a generalist human can. Either version, he says, is really not that far away. He also does not think it will diffuse through society quickly. Amdahl's law still applies: speed up one piece and something else becomes the bottleneck, so the overall speedup is less than you think. Colleagues call it a capability overhang. The tech arrives; society's ability to use it productively lags. The gating factors are people, processes, co-designing interfaces with the way startups actually use models, and social acceptance. A model that invents a brand-new way of doing X still has to wait for humans to make their peace with it.

That lag is, in his biased view, a startup opening. Labs may get to the models first; filling the gap between those capabilities and what end users actually want is likely the company-building job. Even in a world with AGI, human relationships still matter, and knowing your customer will be more important than controlling an artifact that many other labs will also have. How we use computers in ten years is not a replacement of old tools. The command line stays, the GUI stays, voice and ambient computing get added, and agents synthesize interfaces. The metric to watch is leverage per unit of human energy spent with computing, a curve he expects to keep going up and to the right. Specialized models will exist, but mostly for policy rather than technical reasons: a big bank's sales-and-trading and investment-banking AIs should not share information even remotely through their weights, any more than the human employees can today.

He does not believe you simply pull in the cluster from two years from now and everything magically works, and he does not think AGI requires replacing gradient descent or waiting on a quantum computer. Data labeling still has two real jobs: cloning high-quality human behavior to elicit the basics, and teaching the model what good and bad look like on fuzzy tasks. The middle chunk of spamming labels to marginally improve something the model can already do is becoming RL's job. Robotics, in his view, gets a training-wheels version of the reliability problem from digital agents: rearrange a warehouse in digital space first, tune the algorithms on simulated data, then go physical. World models excite him for the remaining hole. Theorem proving has a verifier. Digital agents can have a simulator you boot in staging. The hard case is domains with neither, and world modeling is how he wants to answer that. The thing that would give him the most confidence on robots in the home is the same 100-percent-task recipe in digital space; the bottleneck on many problems, he repeats, is diffusion, not the model.

Amazon, he says, is super serious about generally intelligent systems, especially agents, and people there understand that computing itself is changing. A call to a large model or a large agent may be the most important compute primitive of the future. With Pieter Abbeel he has started a new San Francisco research lab because people at the highest levels of Amazon believe new research breakthroughs are required for the remaining problems on the path to AGI. He pays attention to alternative architectures when they look like they might map learning onto compute more efficiently, but he spends more time looking at data centers and chips. On evaluating new models, two habits: when a published method is simpler than how you used to do it and the results are better, it almost always enters the deep-learning canon; and public benchmarks have been so gamed, and so poorly aligned with what people actually need, that measurement deserves more prestige than it gets. Labs trust internal evals. That is why a model can win coding leaderboards while practitioners still prefer someone else's.

Asked what he changed his mind about in the last year, he keeps coming back to team culture. Hiring really smart, energetic, intrinsically motivated people earlier in their careers is one of the best engines for progress, because every couple of years the optimal playbook in this field changes, and people overfit to the last one slow you down. The other change is more uncomfortable. He used to believe AI capability would compound: win text, then obviously win multimodal, then reasoning and agents. In practice he has seen very little compounding. People are trying relatively similar ideas. Being first to a GPT-4-level model is correlated with being first to a reasoning breakthrough; it is not deterministically true. On NVIDIA, he calls Jensen Huang a complete legend. The under-discussed moves, in his view, were bringing interconnect in-house and orienting the business around systems.

When he joined OpenAI in 2017 the company was a little over a year old and had 35 people. What it got right from the beginning was blurring the lines between research and engineering, which is the function he was hired to run. Around the table were people like John Schulman, Chris Berner on supercomputing, and Wojciech Zaremba. The deeper bet was realizing earlier than everybody else that the era of you and your three best friends writing a paper that changes the world was over. Major scientific goals would be solved by bigger mixed teams of researchers and engineers, whether or not academia scored the solution as novel. When GPT-2 came out, people said it looked like a Transformer. "Yes, it is a Transformer," and that was something to be proud of. Credit for coalescing around that paper goes to Ilya Sutskever, who kept telling colleagues the work was important and poking them to retry experiments on other architectures with the Transformer.

The winning hiring recipe was the same one he still uses: intrinsically motivated people early in their careers. Alec Radford and Aditya Ramesh, who invented DALL-E, did field-defining work without PhDs and without a bajillion years of experience. One unnamed teammate had moved to the Bay, and in a one-on-one about a month and a half in mentioned that he had never bothered setting up Wi-Fi or electricity in his apartment. He just spent all of his time at the office running experiments, and it was completely okay. In the quickfire, Luan expects this year's model progress to look about the same on the surface and actually be more. Overhyped is "scale is dead"—we're totally screwed, let's not buy any more chips. Underhyped is how to solve extremely large-scale simulation for these models to learn from.