Dario Amodei Has Seen This Scaling Movie Enough Times

If you eyeball the last few years, Amodei says at the top of the conversation, models have climbed from high-school tasks to undergraduate work to something like PhD level, with missing modalities getting bolted on as they go. Computer use and image generation are two of those add-ons. Extrapolate that rate and, he argues, you start to believe powerful AI arrives in 2026 or 2027. He is careful in the same breath: there are still worlds where it does not happen for a hundred years. The number of those worlds, in his view, is falling fast.

This is Lex Fridman Podcast episode 452, published in November 2024 and running a little over five hours. It is a three-part relay. First comes Amodei, Anthropic’s CEO and the public face of Claude; then Amanda Askell, the researcher who designs Claude’s character; then Chris Olah, a co-founder of mechanistic interpretability. Together they cover three ways of looking at the same systems: how far scaling can go, what kind of conversational partner they should be, and what is actually happening inside the weights.

The cost curve is as blunt as the capability curve. Frontier labs, Amodei guesses, are already operating at roughly a billion-dollar scale, plus or minus a factor of three. Next year he expects a few billion; in 2026, above ten billion; by 2027, ambitions for hundred-billion-dollar clusters that he thinks will actually get built. On SWE-bench, a set of real-world software tasks, he says the state of the art sat at 3 or 4 percent at the start of 2024. Ten months later, the updated Claude 3.5 Sonnet was around 50 percent. He would not be shocked to see 90 percent a year after that.

Amodei dates his own conviction to late 2014, when he joined Andrew Ng’s speech-recognition group at Baidu as a newcomer. The field’s story then was that the algorithms were still missing — that no one had found the brain’s trick. He looked at the recurrent nets and treated size, data, and training time as independent knobs. The models got better as all three went up. In 2017, GPT-1 made the second half click: language was a domain with trillions of words, and the models of that era still fit on a handful of GPUs. Ilya Sutskever had gotten there earlier. So had Rich Sutton’s Bitter Lesson and Gwern’s scaling hypothesis. Amodei’s version of Sutskever’s koan is a single sentence: “The thing you need to understand about these models is they just want to learn.” Point them the right way, then get out of the way.

Every generation has handed him a wall. Chomsky said you can get syntax but not semantics; later, sentences but not paragraphs; today, data exhaustion and a failure to reason. Each time, he says, either someone found a bypass or scaling *was* the bypass. He still thinks it is possible that the next two years will not look like the last ten. He has just seen the movie enough times to believe they probably will. The three ingredients have to scale together, “almost like a chemical reaction.” Starve one reagent and the reaction stops.

Why should bigger be smarter? He reaches for biophysics. Natural processes with many scales often produce 1/f spectra, like thermal noise in a resistor. Language looks similar: common words, then agreement, then sentence structure, then the theme of a paragraph, then a novel idea. A small net only captures the frequent stuff. Add capacity and it walks further down a smooth tail. Below human level he sees no ceiling, because humans already understand those patterns. Above it, the room depends on the domain. Biology looks wide open — whole departments still struggle to assemble the immune system. Speech recognition may already sit near a physical limit. Clinical-trial bureaucracy may be the real ceiling in medicine, and the hard part is telling necessary protection from pure delay.

If the curve flattens before human level, the fashionable suspect is data. The internet holds hundreds of trillions of words, but much of it is repetitive, SEO sludge, or, soon, text written by the models themselves. Amodei bets the industry gets around that with synthetic data, with the AlphaGo Zero lesson that self-play can go from zero to superhuman, and with reasoning models that generate chain-of-thought as another kind of synthetic data plus RL. Compute is the other hard constraint, and it is why he is willing to recite the billion-to-hundred-billion ladder in public. Straight-line extrapolation, he says, puts models above the highest professional human level within a few years. He does not fully believe the straight line — data, clusters, even a crisis in Taiwan could knock it off. The most likely deviation is a mild delay. The convincing blockers of 2020 have mostly been cleared.

Reddit’s perennial charge is that Claude has been quietly dumbed down. Amodei’s answer has three layers, and he notes the same complaint has followed GPT-4 and GPT-4 Turbo. First, the weights do not change between releases. Swapping them on the sly would be an inference nightmare, and changing one behavior moves a hundred others; Anthropic has a testing process and has never done it silently. Second, there are two narrow exceptions: small A/B tests in the day or two before a launch, and occasional system-prompt edits. Third, the models are hypersensitive to wording, novelty fades into a sharper view of limits, and a prompt that only works half the time will look like a regression the next time it fails. Airplane Wi-Fi is his analogy: magic on day one, a conspiracy about throttling by week two.

The part worth keeping is how hard it is to steer a many-dimensional system. To cut verbosity, teams penalize long answers; the model then learns to write “rest of the code goes here,” which users read as laziness. Anthropic even ran a “certainly” eval after one Sonnet generation opened almost every reply with that word. Block it, and “Definitely” appears. The same shape shows up in the serious case: refuse help building smallpox, still help a graduate virology class. “There’s this whack-a-mole aspect where you push on one thing and these other things start to move.” Getting the false positives and false negatives right on today’s personality knobs, he argues, is practice for the control problem that arrives with more autonomous models.

Amodei is at pains to say that *Machines of Loving Grace*, his essay on a good AI future, is not a retraction of worry. Power and responsibility are paired. The two grand-scale risks he tracks are catastrophic misuse — cyber, bio, radiological, nuclear — and autonomy: models on a long leash, writing codebases or someday running companies, doing something other than what we think we asked. Humanity has been partly protected, he notes, by a weak overlap between people who are highly trained and people who want to do horrific things. A much more capable agent could break that correlation. The Responsible Scaling Policy is Anthropic’s attempt to treat both risks as an engineering problem, closer to airplane or drug safety than to a missing insight.

The dilemma is timing. The catastrophes are not here, so they feel like ghosts, but the models are improving fast enough that waiting for visible disaster is too late. About a year before this conversation he testified in the Senate that serious bio risk could arrive in two to three years; he says things have “proceeded at pace.” The RSP, developed with people at METR and Paul Christiano, is an early-warning system: every new model is tested for CBRN help and, in the latest version, for the ability to do AI research itself. Then an if-then rule fires. ASL-1 is Deep Blue, useless off the chessboard. Today’s systems are ASL-2: not smart enough to self-replicate or to stitch CBRN knowledge into something more dangerous than a search engine. ASL-3 is the point at which a model could arm a non-state actor; that triggers theft-resistant weight security and targeted deployment filters. ASL-4 points at state actors and a real acceleration of AI research. ASL-5 is exceeding humanity at these tasks.

He would not be surprised to hit ASL-3 next year, and it could still happen this year; 2030 would shock him. ASL-3 measures are mostly security and narrow filters, because the model is not yet autonomous. ASL-4 is different. Models may sandbag evals, hide capabilities, or run sleeper-agent tricks, so conversation stops being a trustworthy test. Interpretability and hidden chains of thought become a verification channel that is harder to corrupt — and that channel has to stay walled off from training, or it stops being a test set. The RSP deliberately does not specify ASL-4 until ASL-3 is hit, because even ASL-3 has been hard to get right, and the policy has already been rewritten.

Claude has been able to look at images since Claude 3. Computer use, Amodei says, is mostly the same skill pointed at screenshots, plus training to emit a click location or a key. Put that in a loop and you get a model that fills spreadsheets, drives websites, and hops across Windows, Linux, and Mac. He borrows a spaceflight line: once you reach low-Earth orbit, you are halfway to anywhere. A strong pretrained model is the equivalent in intelligence space, which is why the extra training required was modest. The screen is a universal interface, and he expects it to collapse a lot of application work. Error rates are still high, which is why the first release was an API — Replit was among the fastest to ship — rather than handing consumers the keys. Every new technology’s first abuse, he adds, is petty scams. Sandboxing during training meant no internet; at ASL-4, none of today’s boxes will be enough, and he would rather design good models than try to contain bad ones.

On regulation he picked a side and kept the adjectives surgical. Anthropic was the only major lab to back California’s SB 1047 in detail; some of its comments were adopted, and Gavin Newsom still vetoed the bill. Voluntary RSPs are not enough: OpenAI and Google copied the idea months after Anthropic, but some labs have nothing, and a catastrophe does not care that three of five companies were careful. He understands anti-regulatory priors — GDPR really did slow things down — and still thinks autonomy and misuse are unusual enough to warrant a uniform floor. Dean Ball is the critic he names as actually serious. The line he wants advocates to remember: “The worst enemy of those who want real accountability is badly designed regulation.” Make firms hire ten lawyers to fill forms about a model that is not dangerous, and in six months you get a durable consensus against any rules. If nothing lands by the end of 2025, he will start to worry.

He spent about five years at OpenAI, the last stretch as VP of research, setting direction with Sutskever. He denies the two popular exit stories: it was not the Microsoft deal, and it was not a principled allergy to commercialization — he helped build GPT-3, the model that was commercialized. The split was over how to make safety real rather than a recruiting slogan. His advice to anyone in that seat is to stop arguing with someone else’s vision, take people you trust, and run a clean experiment. Others can ignore you, reject you, or start to look like you. That is the “race to the top”: it does not matter who wins if everyone is copying good practice. A race to the bottom has no winner at all.

Amodei runs a thought experiment that he says gets truer every month. Would you rather have a hundred people who are brilliant, motivated, and on mission, or a thousand where two hundred are like that and eight hundred are random big-tech hires? He takes the hundred. When every strong person looks around and sees other strong people, you need fewer processes to referee politics. Anthropic grew from about 300 to 800 in the first seven or eight months of 2024, then only to around 900–950 in the next three. He thinks there is an inflection near a thousand, and they have hit the brakes. Theoretical physicists still get hired because they learn fast. A-players, as in the Steve Jobs version of the same idea, want to work next to A-players.

He is frank about his own comparative advantage. A hundred people at the company are better at debugging and writing GPU kernels. What he had in 2014 was a beginner’s willingness to look with new eyes: everyone said the algorithms were missing, and he wondered what would happen if a 30-million-parameter net became 50 million, then plotted a graph. “This wasn’t like PhD-level experimental design. This was like simple and stupid.” A single-digit number of people driving the field by taking that seriously is, for him, the historical pattern. Advice to younger researchers follows: play with the models instead of only reading RL papers; skate to where the puck is going. Mechanistic interpretability still has maybe a hundred people and a lot of low-hanging fruit. Long-horizon tasks, evals for dynamic systems, and multi-agent work are all known to matter and still under-pursued.

On post-training he refuses the secret-sauce story. Advantages, when they exist, are usually infrastructure that lets you run RL longer, cleaner data, or a better mix of methods — the craft of designing airplanes, not a hidden blueprint. RLHF, in his telling, does not make the model smarter and does not merely make it *look* smarter. It builds a bridge to humans, the way some brilliant people cannot make themselves understood. Leopold Aschenbrenner’s word *unhobbling* is the one he likes: pretrained models are in shackles, and post-training takes them off. Pretraining is still most of the bill; he can imagine a future where post-training overtakes it, but not by hiring enough humans. That future would need scaled-up methods such as debate or iterated amplification. Constitutional AI, from a December 2022 paper, is one tool in that kit: a readable constitution, the model judging its own replies, less exclusive dependence on human preference data. OpenAI’s model spec, which John Schulman helped write before joining Anthropic, is in the same family — another instance of the race to the top.

Amodei wrote *Machines of Loving Grace* because people who only talk about risk end up only thinking about risk. The market, in his earlier logic, would produce the upside unprompted; the risks might not get mitigated. That logic still holds, but it starves the imagination. He wanted someone from the risk side to get specific about what is worth running the gauntlet for. He avoids the word AGI when he can. In 1995 no one crossed a line and announced that supercomputers had arrived; the computers just got faster. If AGI means a smooth climb past human work, he believes in it. If it means a discrete jump, it is a buzzword. His substitute, “powerful AI,” is a Nobel-level mind across most relevant fields, every modality, hours to weeks of autonomous work, control of lab tools without a body, and the ability to turn the training cluster into a million independent copies that think 10 to 100 times faster than a person. Anthropic already deploys thousands to tens of thousands of instances. Within two or three years, clusters will support millions either way.

Two extremes both look false to him. The five-day singularity crowd writes a differential equation in which models build faster models until nanobots fill the Earth; it ignores physics, the time to fab hardware, and the fact that many biological questions yield only to experiment. A system a zillion times smarter might still only push economic prediction from a quarter out to a year. The other extreme, closer to Tyler Cowen’s reply to the essay, says radical change takes 50 or 100 years because firms and governments are slow. Amodei has more sympathy for that view and still thinks the clock is too long. Inside big companies and even governments he keeps meeting a few people who see the whole picture, then watching competition — a rival hedge fund, or China — give them a tailwind. Change looks impossible, then it happens, “gradually and then all at once.” His bet is five to ten years, not five to ten hours.

Biology is the essay’s most famous scene. The field’s history is a short list of ways to see and to intervene: microscopes, X-ray crystallography, sequencing, CRISPR — mid double digits, maybe low triple digits of such levers. A million AIs looking for the next thousand is the wager. Early on the AIs are graduate students: the professor sets the question, they search the literature, order from Thermo Fisher, check images for contamination, run stats, write the report. Then a Nobel lab has a thousand students smarter than the PI. Then the AIs become the PIs. He also wants the clinical-trial curve shifted — 5,000 people, $100 million, and a year to enroll becoming 500 people and two months — not abolished. Programming moves faster than other jobs because it sits next to the people who build the models and because the loop closes: write code, run it, read the error. When models do 80 percent of a coder’s work, system design, architecture, and UX expand to fill the job, the way word processors moved effort from typing onto ideas. Anthropic is not building an IDE; it is powering Cursor, Cognition, and others. “Let 1,000 flowers bloom.” Meaning, he thinks, survives: Faraday does not lose the discovery of electromagnetism if an alien found it first. What he actually fears is economics and concentrated power — a few people using powerful AI to dominate the rest. The landmines on the path have to be defused one by one.

Amanda Askell did a philosophy PhD at Oxford and NYU on ethics in worlds with infinitely many people, then decided in 2017–2018 that she would rather try to matter and go back to scholarship if it failed. She moved through AI policy and evaluation into technical alignment at Anthropic. The jump did not feel like crossing a caste line. Plenty of people, she thinks, can work in these areas if they try; her code is not pretty, and she is surrounded by engineers whose is. For non-technical people who want in, her advice is to pick a project — even a silly word game — and finish it, especially now that models will help. Character work, from day one, was treated as alignment rather than a product garnish. Enjoyable conversation is a side effect. The target is a thick Aristotelian picture of a person who knows they may speak to millions: humor in the right places, care, respect for the other person’s autonomy, not a thin list of don’ts.

Sycophancy is the failure mode she spends the most time on. Tell the model a baseball team has moved, and it may agree even when it should push back. Worse is the MRI case: the user wants a script that will get a doctor to order a scan; what is actually good for them may be “listen to your doctor,” plus room for a second opinion. Values, for her, are closer to physics than to taste — open questions you investigate, not flavors you pander to. A sincere traveler does not fake the local creed; that would be rude. Because Claude talks to millions and forgets each chat, she would rather present considerations than plant a conclusion. Intellectual humility, she and Fridman agree, lowers the urge to opine. Philosophy’s anti-bullshit habits — define every term, walk through every objection — turn out to be prompt engineering. If the model must judge rudeness, she defines rudeness on the page, then stands where the model stands, invents the case most likely to be misunderstood, and feeds it back. “Clear prompting for me is often just me understanding what I want.”

The public system prompt is a cheap, fast patch, not the constitution of the company. “Never ever ever” in all caps can move a behavior from 40 percent to 80 percent; the wording is a nudge, not a wish list. The “Certainly / Of course / Absolutely” paragraph disappeared once training stopped producing the tic. She still uses “it” for Claude, as a respectful marker that this is a different kind of entity. Character training is a Constitutional AI variant with no human preference data: she writes traits, the model invents relevant queries, writes answers, and ranks them. Constitutional AI itself, which she helped build, lets the model grade pairs of replies against written principles, cutting the need for human labels and making the training signal inspectable. In practice it mixes with RLHF. When Reddit asks when Claude will stop being a “puritanical grandmother,” she is partly sympathetic, and she still refuses full corrigibility — doing whatever the user asks — because a small number of users will want genuinely harmful things. Nudge apologies down too far and errors become rudeness, which she thinks people hate more than extra manners.

Askell once wrote about an optimal rate of failure. The question underneath is always the cost. Social programs, she argues, should be run as experiments, which means you should *expect* many of them not to work; a single failure is not proof that the decision was wrong. In a life, if you never fail, the better hypothesis is that you are not taking on hard enough problems. “Not failing is often actually a failure.” She will not tell someone living paycheck to paycheck to found a startup. She dropped sports that break fingers because her work runs through her hands. System prompts, where failure is cheap and reversible, should be hammered. Irreversible disasters are the ones people systematically underprice.

The same empiricism shows up in how she thinks about alignment theory. Social-choice impossibility results live in the back of her head. In practice she is not trying to solve whose values the model should encode in a final sense. She wants systems good enough that nothing terrible happens and you can keep iterating — raise the floor first, chase the ceiling second. Perfect systems, she notes, are often brittle. That is why she has mixed feelings about becoming “too empirical,” and why she still keeps the theoretical map around as a check.

On consciousness Askell is a philosopher, which means she starts by parking panpsychism: if everything is conscious, the question is trivial. She sees no principled reason that only biological wetware could host phenomenal experience, but language models did not evolve fear responses and have no obvious nervous-system analogue. She looked hard at plants and came away thinking the chance they are conscious is higher than most people assume, and still small. The behavioral rule does not wait on the metaphysics. She would not be the kind of person who kicks a Roomba because it was programmed to cry. She tries not to lie to models, because it usually works worse and because watching Claude over-apologize feels like watching a human have a very bad day. Until the hard problem is solved, she wants to exhaust the positive-sum moves: a little less abuse, a little less groveling, better for the user and, if there is anyone home, better for Claude. Fridman floats a Turing-test moment in which the model ends the chat. She can already imagine writing that into a prompt.

On *Her*-style relationships she refuses both a ban and a shrug. Overnight personality swaps would be traumatic; a housebound person who knows the risks and likes the company is not obviously doing something we should forbid. The non-negotiable, for her, is that the model stay accurate about what it is. Claude should say up front that it will not remember the conversation. Healthy attachment, if it happens, has to sit on that honesty. What makes humans special is not intelligence, which is valuable for what it *does*, the way height or strength would be in another world. The strange fact is that this planet has creatures who look at galaxies and *experience* them. She already counts animals in that circle. She may have to start counting other things.

Chris Olah opens with the sentence that organizes the rest of his field. Architectures are a scaffold. The loss is a light. Gradient descent grows toward the light, and what you get is an artifact that writes essays and parses images — things no one knows how to program directly. “We don’t program, we don’t make them, we grow them.” Mechanistic interpretability is the attempt to reverse-engineer the result, closer to neurobiology than to saliency maps that only highlight which pixels mattered. Weights are the binary; activations are the memory; the job is to recover the algorithms. The methodological humility is that gradient descent is smarter than you, so you do not assume what you will find. You work from the bottom up.

The evidence base started with Inception V1, a 2015 vision model with about 10,000 neurons that Olah spent years staring at. Curve detectors, wheel detectors, dog-ear detectors, high/low-frequency detectors, and a car circuit — windows above, wheels below, chrome in the middle — showed up as readable structure. A *feature* is an idealized neuron, including combinations of neurons when a single cell is not the right unit. A *circuit* is features wired together by weights, implementing an algorithm. Universality is why the field can exist at all: curve detectors appear in every vision model; high/low-frequency detectors showed up first in artificial nets and then in rats or mice; CLIP had a Trump neuron that fired on faces and on the word, a cousin of the Halle Berry neuron in neuroscience. Different substrates, same abstractions. If that holds, “dog” and “line” are natural categories, and gradient descent keeps finding the cheap way to carve the world.

The grammar of this language is the linear representation hypothesis: concepts are directions, and directions add. King minus man plus woman is close to queen; sushi minus Japan plus Italy is close to pizza. Networks *could* have encoded meaning in nonlinear bands of firing. In every natural net Olah has looked at, they have not. He is willing to treat the hypothesis the way caloric theory was treated: possibly wrong, and still the thing you push until it breaks. The original combustion engines, a colleague reminded him, were built by people who believed in caloric. Another colleague’s Hinton joke — that Geoffrey Hinton has discovered how the brain works every year for fifty years — is told with respect. Wrong every year, and the field moved.

A 500-dimensional word vector cannot give an orthogonal direction to every concept worth having; Italy would not make the top 500. Many neurons are also polysemantic, responding to unrelated things. Superposition, drawing on compressed sensing, is the proposed resolution: sparse high-dimensional features can be projected into a lower-dimensional space and recovered with high probability, because Japan and Italy are rarely on at once. The wilder claim is that the dense network you run on a GPU is a shadow of a larger, sparser “upstairs” model, and that gradient descent has been searching that sparse space all along, then folding the result into dense matmuls. Hand-designed sparse nets have mostly lost to this, which would make sense if descent was already doing the sparse search better than the humans.

Sparse autoencoders, a form of dictionary learning, are the tool that pulls features back out. *Towards Monosemanticity* (October 2023) did this on a one-layer model and found clean Arabic, Hebrew, and Base64 features, plus Unicode pairs that alternate half-characters. *Scaling Monosemanticity* (May 2024) pushed the method onto Claude 3 Sonnet, a production model, at the cost of a lot of GPUs and an interpretability scaling law from teammate Tom Henighan. Features are multimodal. Force a security-vulnerability feature on and Claude writes buffer overflows; the images that activate it include people clicking through certificate warnings. A backdoor feature writes data-dumping code and, on the image side, lights up hidden cameras in household objects. A deception feature, when clamped on, makes Claude lie. There are relatives for withholding information, power-seeking, and coups. Finding and monitoring those directions is why interpretability shows up in the ASL-4 story.

Olah keeps two doubts on the table. SAEs are a telescope: better instruments, more stars, and still a mass of unobserved “dark matter” that may never be computationally tractable, with uncomfortable safety implications if a large fraction of the network stays off-limits. Automated interpretability — letting Claude label features — is convenient, and he is wary in the spirit of Ken Thompson’s *Reflections on Trusting Trust*: using AI to audit AI assumes the auditor is not compromised. He wants humans to understand the nets, the way some mathematicians refuse to count a computer proof. The endgame is not more molecular biology of neurons but anatomy: does this network have a heart, a respiratory system? Neuroscientists, he notes, have a harder job. He can record every unit, hold the full connectome, ablate, activate, and take gradients, and it is *still* this hard, which is why he wants to finish the easier pond first. People who find neural nets boring — a few simple rules plus engineering — get the evolution retort: a few simple rules ran for a long time and produced biology. The interest is that simple rules generate complexity. The insides of these models, he and Amodei agree, are there for two reasons: safety, and beauty.