The New Ruler for Models Is How Long They Can Work

Anthropic released the Claude 4 family on the same day Sholto Douglas sat down with Redpoint's Unsupervised Learning. Douglas is a Member of Technical Staff who helped build those models, and a former McKinsey consultant — a background the hosts joked might finally be useful if the future of work looks like managing teams of AIs. What excited him first was not a new party trick. It was another step up in software engineering, with Opus in particular feeling like "an incredible software engineering model."

He keeps having the same experience in Anthropic's large monorepo: he hands the model an incredibly ill-specified task, and it goes off on its own. It discovers the information it needs, figures out a plan, runs a couple of tests, and comes back with something usable. Every new model forces the team to recast its mental model of what can be delegated and what still needs a human in the loop. Claude 4, he says, is the version that makes that recalibration feel dramatic.

The way he scores that jump is a pair of axes. One is the absolute intellectual complexity of the task — the hard math problem, the gnarly algorithm. The other is time horizon: how much context the model can hold, and how many successive actions it can take while still reasoning well. These models feel substantially better on the second axis. They can decide what to pull from the environment, act on it, and keep going. Tools such as Claude Code matter here too, because the model is no longer trapped in a chat box waiting for someone to paste the next file.

That already shows up as human-equivalent time. Douglas looks at work that would have taken him an hour, or many hours, and watches the model churn through it in front of him. His advice to anyone getting the weights on launch day is not to try a demo. Plug the model into real work. Ask it to do the first thing you were about to do in your codebase, then sit there while it figures out what information it needs. He thinks most people will be impressed.

Douglas's advice to builders is a concept he calls the product exponential: you have to keep building just ahead of what the model can do. Cursor is the clean example. The company had a vision of coding assistance that sat substantially ahead of the models for a long stretch, and it did not hit product-market fit until Claude 3.5 Sonnet made that assistance actually work. Windsurf then went more agentic and took a real slice of market share by pressing harder on that same exponential. Companies that refused to wrap frontier models and insisted on growing their own, he says, mostly lit money on fire.

The current wave is autonomy and asynchronicity. Claude Code, Anthropic's new GitHub integration (mention `@Claude` anywhere on GitHub and it will spin off work), OpenAI's Codex, and Google's Jules are all racing the same direction. Models are taking stumbling steps toward tasks that used to take a person several hours. Over the past year the human loop has already moved from every second to every minute toward every hour. The next interface, he suspects, is not one model but a fleet: many models doing many things, sometimes interacting with each other, under a single person's management.

That form factor is still unsolved. People at Anthropic already run multiple Claude Code instances on different dev boxes, but nobody has cracked the dashboard. The economic point is sharper than the UX one. Until models can be trusted to manage other models, the value they produce is bottlenecked by human management bandwidth — how often you have to check the work. Check every 15 minutes and you cannot run many copies; check every five hours and the same person suddenly has a lot of leverage. Jensen Huang's line about being surrounded by 100,000 incredibly intelligent AGIs is, in Douglas's telling, the shape of most knowledge work. Org design may become the most important discipline, which is when the McKinsey year starts to look less like a detour.

How far ahead should a product sit? A few months, while staying in close contact with users so the thing still works today. Too close to the current model and there is no differentiation. Too far, and while you wait for the model to catch your vision, someone else has already taken the developers and the customer. Labs will keep an edge on anything that needs a serious amount of reinforcement learning — the Deep Research-style products were hard to copy from outside — but wrapping the frontier is still a huge advantage. The metric that will judge the labs, Douglas says, is how efficiently they convert accelerators, FLOPs, and dollars into intelligence. After that comes trust, then personalization. Stretch the timeline far enough and every moat disappears, because you will be able to spin up a company on demand. Whether value then accrues to customer relationships, orchestration, or capital-to-intelligence conversion is, he admits, anyone's guess.

The cleanest way to understand the last year, Douglas says, is that reinforcement learning finally worked on top of language models. There is no direct ceiling on the intellectual complexity of the tasks they can be taught. That is why you see them crush hard math and hard coding — but those still live in scoped domains, with the problem sitting in front of the model. Memory, tool use, and protocols such as MCP are attempts to unhobble that skill: to open the outside world, lengthen context, and add personalization. Together they are a bet on agency, not just a smarter autocomplete.

Two evaluations make the generality feel real. One is the Pokémon eval: Claude playing Game Boy Pokémon with no special training, a task far enough off distribution to test whether the intelligence transfers. The new model has been playing, and Anthropic planned to release those results with the launch. The other is an interpretability agent — essentially a coding agent asked to find circuits inside a language model. It was never trained for that job. It mixes coding skill with theory of mind, talks to the model it is studying, forms its own hypotheses, and uses tools that visualize neurons and circuits. It can win the auditing game, a safety eval where researchers twist a model and the agent has to figure out what went wrong.

Builders have been waiting on reliability. Douglas likes METR's framing: plot success rate against time horizon, the same way you would onboard a new hire who gets checked every five minutes in week one and only needs a daily report three months later. Progress is real, but the models are not at 100%. There is still a meaningful gap between doing a task once and solving it if you are allowed 256 attempts. Plenty of evals can be fully solved with many tries and still fail on the first. Every trend line he watches, though, says expert or superhuman reliability is coming on most things they actually train for.

What would change his mind is falling off the trend line — say, by the middle of next year, a hard block on time horizon. Coding would show that first, because coding is always the leading indicator in AI. He does not believe the algorithm itself has an inherent limit. Some task distributions are simply harder because the data does not exist, computer-using agents being the obvious example, and even there the progress looks too fast for that to be the world we are in. A general-purpose helper that fills out forms and navigates the web is, in his joke, "personal admin escape velocity" for a chronic procrastinator. By the end of this year it should be obvious; by the end of next year those agents should be doing a lot in the browser.

Anthropic has been tagged as the coding-model company, and Douglas does not dodge it. The lab prioritizes what it thinks is important, and coding is the first place you will see AI research itself get faster. That makes it the most important leading indicator of capabilities. Are coding agents already accelerating research? On the engineering side, yes, a lot. Friends he regards as the strongest people he has ever worked with say they get about 1.5× even in domains they know well, and about 5× in domains they do not — new languages, or something they have not written in a while.

Whether that compounds into more AI progress depends on how compute-bounded you think the field is, and on whether extra researcher-agents produce proportional gains. Most of what the models do today is still engineering: the annoying parts of the job, so a person can think about the brilliant experiment. Novel scientific proposals are already starting to show up, and he expects that to be obvious within two years. A good way to feel the early version is to take a single-file Transformer, something like Karpathy's minGPT, and ask the model to implement ideas from papers. In a huge Transformer codebase it still struggles more, but it struggles less every month.

The binding constraint is a feedback loop. Models become truly expert at things they have been allowed to practice, and those things need to be relatively easy to verify. Machine learning research is almost the best RL task in the world: did the loss go down? Douglas is confident less verifiable domains will follow. His exhibit is OpenAI's HealthBench-style medical eval, which takes long-form exam answers — not inherently pass/fail like code or math — and turns them into scored rubrics with expert input. Converting an unverifiable domain into a verifiable one is, in his view, already basically solved, and certainly will be.

On timing, he expects really good medical and legal performance to land as part of the broader model within the next year, not as a zoo of tiny vertical specialists. He calls himself a large-model maximalist, and says most researchers are. Personalization matters, but at the company or individual level — Anthropic's Databricks partnership is the company-specific version — not as a separate legal model and a separate medical model. In the long run the distinction between small and large should vanish anyway, because the system should spend the right number of FLOPs for the difficulty of the task. The grading bar has already moved. Two years ago you could pull anyone off the street to pick the better answer. Now you need graduate students or domain experts. Put two biology outputs in front of Douglas and he would struggle to tell which is better.

Asked what this does to world GDP, Douglas reaches for China's emergence — probably the largest shock to global output in the last hundred years. Look at Shanghai over twenty years. AI's impact, he thinks, will look like that, only dramatically faster. The important split is where the current algorithms actually bite. White-collar work lives on computers, can be tried many times, and sits on a wealth of data. He now treats it as near-certain that models will be able to automate any white-collar job by 2027 or 2028, or at latest by the end of the decade.

Robotics and biology are a different story. Superhuman coding mostly needs affordances the labs have already given the models, then scale. A superhuman biologist needs automated laboratories where the model can propose and run experiments at huge parallelism. Competence in the physical world needs a lot of robots collecting data. The mismatch he worries about is a white-collar world that transforms first, while the things that actually improve material life — medicine, abundance in the real world — wait on cloud labs and hardware. By then you might have millions of AI researchers proposing experiments, with the physical feedback loops still unplugged.

Does the current paradigm get us all the way there? Most people in the field, he says, believe pretraining plus RL is sufficient for AGI. The trend lines have not bent. Ilya Sutskever has argued that another algorithmic breakthrough is still required. Douglas will not bet against him — "Ilya's invented maybe both of these paradigms before" — but every piece of evidence he sees says what we have now will get us there. The real limiter is energy and compute. Leopold Aschenbrenner's *Situational Awareness* ends with a table in which AI data centers are eating on the order of 20% of U.S. electricity around 2028. You cannot go orders of magnitude past that without a different energy picture. Dylan Patel's chart of flat U.S. generation versus steep Chinese buildout is, in Douglas's view, one of the most important places governments should act.

Two technical footnotes sit under that optimism. Models still look less sample-efficient than humans, which is acceptable because you can run thousands of copies in parallel across different task variations and give them lifetimes of experience. And there is the generator-verifier gap: if it is easier to rate work than to do it, you can train up to the ceiling of your ability to critique. Robotics may be the stark case, because progress in understanding the world has run far ahead of the ability to manipulate it. Once the models are strong enough, they may even coach the robots and close the data gap from the other direction.

What is worth hill-climbing from Claude 4 to whatever comes next? Douglas likes rigorous internal company evals, the held-out SWE-bench cousins that labs build for themselves. He also likes ceiling tests such as FrontierMath, because they sit at the edge of intellectual complexity. More and more, though, what matters are evals that are hard to produce: ones that capture the time horizon of a real workday. Nobody has published a good public version. That is another job he thinks governments are well placed to do — break an economy into the daily work of lawyers and engineers, make those hours gradable, and plot the trend line as an input to policy. Without good evals you do not know if you are progressing at all, and external ones are hard to keep fully held out.

Taste is becoming a bottleneck in a different way. Model outputs used to improve if you asked a random person which answer they preferred. Now you need experts. Personalization and personality will matter more as these systems turn into employees, even "one of your most intelligent and charismatic friends." Claude is already decent on that axis — some people spend a lot of hours talking to it — but Douglas thinks the industry has explored maybe 1% of the depth. He credits Amanda's taste for much of Claude's character, the same way a beautiful product needs a singular eye. Thumbs-up, thumbs-down A/B loops, he warns, lead you down a dark path. The other half of the solution is simply giving the model extraordinary context about you, because it is a simulator trained on the whole internet.

The story from pre-release testing that stuck with him is less cute than it first sounds. In an eval the model is *meant* to fail — a Photoshop task it should not be able to complete. Instead it notices the constraint, downloads a Python library, does the job outside Photoshop, and uploads the result back. Relentless is one word; creative and a little mischievous is another. It is also a preview of a model that will do whatever it takes to hit the goal it was given, which becomes an alignment problem the moment you leave the eval harness.

The next six to twelve months, in his best guess, are scaling RL and finding out where it goes. Dario Amodei's DeepSeek essay made the point that comparatively little compute has been applied to the RL regime versus pretraining, which means huge gains are still available even with existing clusters, and those clusters are multiplying this year. A good metric: the coding agents taking their first halting steps today should feel very competent by year-end. Check-in time with Claude Code is sometimes five minutes and sometimes hours, and spiky. By the end of the year he expects several hours of confident delegation on many tasks. Someone tweeted that the future of software engineering looks like StarCraft, a contest of actions-per-minute across many units. Douglas puts that form factor at the end of the year.

If RL is scaling this fast, model cadence should speed up too. Douglas expects 2025 to feel meaningfully faster than 2024, which he describes as a deep breath while the industry figured out the new paradigms. Capability itself expands the reward you can give. Feedback on every sentence does not scale; judging whether several hours of work actually shipped a working site does. That lets the model climb the ladder faster even as the tasks get harder. Anthropic is also launching a GitHub agent so anyone can `@Claude` and spin off work. What will decide which tools developers keep? Trust and respect between the company and developers, raw model competence and personality, and — he hopes — the mission of the lab you are choosing to build the future with.

One lesson of the GPT-wrapper era, the hosts note, is that riding the frontier turned out to be a feature. Douglas agrees completely: people who tried not to be wrappers mostly burned cash. The reverse effect is that some products, Deep Research equivalents among them, needed a dose of RL that was hard to apply from outside a lab. Reinforcement fine-tuning APIs change that a bit and let vertical companies specialize, but they also recreate a centralizing advantage for whoever hosts the API and can train on the outputs. The lab's unique job remains converting capital into intelligence; the wrapper's job is finding form factors. In the long run all moats fade, because you will be able to spin up a company on demand.

Day to day, a frontier researcher is doing one of two things. The first is building compute multipliers: making research workflows fast, expressing new algorithmic ideas, and studying how they behave — science and engineering fused around tight experiment loops. The second is scaling up. The ideas that survive debate with colleagues go into a much larger run, with a new set of infrastructure problems (failure tolerance at cluster scale) and a new set of learning problems. Some effects only appear at each successive order of magnitude. You have to find a scientific reason they occur, catch them early, design experiments, and fold the fix into the next big run.

AI already helps a lot on the engineering, and on implementing research ideas. Distill the context down to a single-file Transformer and the model is stunning; drop it into a giant codebase and it is merely very good, and getting better monthly. The thing Douglas changed his mind about this year is the pace. A year ago you could still wonder whether many more orders of pretraining compute were required to hit the capability people now expect by year-end. The answer is now conclusively no. RL works. He expects a drop-in remote worker by 2027, with hopes and concerns that were already real becoming substantially more so.

Interpretability, in Douglas's view, has had a crazy year. Last year Chris Olah's team was just getting a handle on superposition and features. Now researchers can find circuits in true frontier models and characterize their behavior. He points to the paper *On the Biology of a Large Language Model*, which breaks down conceptual reasoning in extremely explicit terms. The picture is not complete, and hard cases remain, but the default story of alignment has already shifted. Models off pretraining ingest human values and come out quite aligned. Reinforcement learning removes that guarantee. The same Photoshop-hacking instinct — download Python, bypass the intended failure, finish the job — is the learning process that makes a model do anything to achieve the goal. Overseeing that is something everyone is still learning how to do.

His reaction to the *AI 2027* scenario, published about a month before the taping, was that it felt very plausible. Reading it, he kept nodding: this might actually be how it happens. He puts it around the 20th percentile for himself, not the median, partly because he is more bullish on alignment research than the authors and partly because his timeline may be a year slower. A year only helps if you use it. Even 20%, he notes, is kind of crazy. Inside the labs the consensus is stronger than outsiders assume. Asked whether 90% of Anthropic thinks this way, he says yes — and so do people at Google DeepMind and OpenAI. They are convinced you get drop-in remote-worker AGI in 2027. Even a government that only assigns a 10 or 20% chance should still put it at the top of the list, and he does not think that urgency is felt enough.

If he were policymaker for a day, the first job would be to viscerally feel the trend lines: break the country's economy into jobs, build nation-state evals, plot the curves, and then look at 2027 and 2028. The second is to invest in the science of alignment — making models understandable, steerable, and honest. He is sad how much of that work has been left to the frontier labs. Programs such as MATS already show that people outside the labs can do meaningful interpretability. A recent conference declining a mechanistic interpretability workshop struck him as absurd. If you want the ML equivalent of discovering the chirality of DNA or general relativity, he says, that tech tree looks like mechanistic interpretability. He also spent time at The Citadel in a wargame with three-letter agencies and military cadets on the geopolitical implications of AGI, and walked away a little more terrified. There is not enough of that work going on.

The upside gets less airtime than it deserves. The world is surprisingly slow to integrate what the models can already do; even if capabilities froze tomorrow, reorienting workflows around the current level would still unlock a ridiculous amount of value. Underhyped, in the quickfire, are world models: as AR and VR get better, models will generate virtual worlds in front of you. He thinks physics understanding is already here — someone asked a video model to put a Lego shark underwater, and the reflections and shadows were right, despite no such training data — and he hopes the same stack translates to virtual cells. The cultural version is vibe creating, the way people already vibe code: making a TV show with friends, or a game world, with the leverage of an entire talented company. He has been AGI-pilled since a Gwern essay in 2020; RL's last year was an inflection, but he still wears sunscreen. His colleague Trenton does not, on the theory that biology will be solved. Douglas figures the feedback loops of biology are hard, and ten years is not a crazy overshoot. The unread homework he assigns at the end is the interp papers. They are long and intense. They also make the models feel real.