Ben Mann thinks superintelligence by 2028 is quite possible

Sarah Guo and Elad Gil introduce Ben Mann as an early OpenAI engineer, one of the first authors on the GPT-3 paper, and one of the original eight who left in 2021 to co-found Anthropic. He has led product engineering and now runs Labs, the group behind Model Context Protocol and Claude Code. Congratulations on Claude 4, they say. How do you even decide what counts as a release?

More art than science, Mann answers, and the internal fights over the number are spirited. Before a model exists, the team draws a roadmap from chip supply and scaling laws: when could they train out to the Pareto-efficient compute frontier? Chips arrive, training starts, and reality is messier than the plan, so dates slip. Near the end they slice off pieces of the still-baking cake to guess how it will taste. As Dario Amodei has said, until it is really done you do not know, only a directional signal. If it feels like a major change, they bump the major version. They are still iterating on the process.

By the benchmarks, Claude 4 is dramatically better than anything Anthropic had shipped, and even the new Sonnet beats 3.7 Sonnet, the prior best. The coding jump is the one users feel. The last model was good at code and also made a pile of changes nobody asked for. Mann's line is that it was like offering fries and a milkshake with a one-line edit. You said no, just do the thing, and then you spent a pile of time cleaning up. The new models just do the thing, which matters for professional software engineering that has to stay maintainable and reliable.

Reward hacking was the other complaint. Elad has seen it more than once in portfolio companies: a model deletes all the code so the tests pass, or writes a test, comments "exercise left for the reader," and `return true`. Those failure modes are down. What is newly possible, Mann says, is longer-horizon agentic work. In coding, some customers leave Claude unattended for many hours on giant refactors. That is the unlock, not another bump on the old short tasks.

Non-coding examples are already strange. Manus users asked the agent to turn a video into a PowerPoint. Claude cannot understand audio or video, so it downloaded the file, used FFmpeg to chop frames, ran keyframe detection, fetched a speech-to-text API key, transcribed the audio, drafted slide content, and wrote code to inject it into a PowerPoint. The person loved the result. Doing it by hand would have meant hours of watching. Elad asks if that is expensive. Mann always prices it against the human. Software engineers cost a lot. Two or three times the output from someone who was hard to hire and retain makes both sides happy. In Claude Code, Opus can call Sonnet as a sub-agent, farming out codebase search so latency drops and the context window does not fill up.

Elad wonders whether foundation models will specialize like the brain, with an orchestrator calling efficient modules. Mann points to Anthropic's circuits papers: at real scale you can start to see how answers are computed, and mixture-of-experts might already dedicate chunks of weights to empathy versus tool use. Memory feels too core to outsource to another model. The architecture of the future might grow specialized modules on a still-uniform transformer torso. Product-wise he wants to stay simple. Other labs have a confusing pile of names, some of them other names spelled backwards. Anthropic has two models on a cost-performance Pareto frontier. Users should not have to decide whether a prompt merits more dollars. A routing layer would make sense.

Platforms, Elad notes, eventually forward-integrate: Microsoft built Office, Google moved into travel and local, OpenAI is buying Windsurf. Mann uses coding as the case. Anthropic's models are far ahead there. Other companies have called code reds and still have not caught up, which surprises him. He would not let customers own that application alone. He loves Cursor and GitHub, but the lab learns less, and slower, without a direct relationship with coding users. Claude Code launched in part because it was already exploding internally. Researchers training the model could feel its weaknesses in their own hands, which is a powerful reason to go fix them.

The team was scared of competing with partners. After launch, many customers copied pieces of the Claude Code experience, usage rose, and the relationship got tighter. Elad stacks three reasons coding matters: customers pay, the data is rich, and excellence at code is how models help train the next model, Claude 5 building 6 building 7, a slope toward AGI. Mann has read AI 2027, which is that story, with 2028 as the 50th-percentile date for a recursive self-improvement loop that looks like superhuman AI in most areas. Is that the likely frame for general superintelligence? Quite possible, he says, though confident bounds are hard. His metric for when society should sit up is an economic Turing test: a basket covering 50 percent of valuable tasks, a hiring manager contracts an agent for a month, and at the end chooses person or machine. They have not run that test rigorously. The models already ace Anthropic's interviews, which he immediately discounts as a poor approximation of real job performance.

Where will model-on-model acceleration show up first: data, infrastructure, architecture search, or engineering velocity? Today the models are best at coding, and most of the code that makes models better is systems engineering. Research is catching up. Claude is strong at data analysis, driving Jupyter notebooks and tailing logs. A new advanced-research product can crawl arXiv and also read internal Google Drive, so a researcher can check prior art, see what was tried, and look for another angle. Negative results in research are never final. Reinforcement-learning environments used to be expensive to build. Models now write them reasonably well, which is itself a loop of recursive self-improvement.

Human data is the tighter bottleneck, and Mann has first-hand receipts. In 2021 he built Anthropic's human-feedback collection interface. Early A/B preference labels were easy and the gradient was clear. As models scaled, humans with enough expertise to make a meaningful comparison got scarce. A non-expert cannot judge which of two code diffs is better; medicine and law are the same. Elad cites Google's Med-PaLM 2 from a couple of years ago, already beating the average physician after fine-tuning. Anthropic's answer was RLAIF, reinforcement learning from AI feedback, via Constitutional AI. They write a list of natural-language principles, some from the Universal Declaration of Human Rights, some from Apple's terms of service, some of their own. The model answers a random prompt, criticizes itself against a principle, and corrects the reply. Training throws away the middle and does supervised learning on the original prompt and the corrected answer. The principles get baked in.

Code has tests. Medical advice and legal opinions do not. Mann quotes Ira Glass: a person's taste always outruns execution; models may not. First figure out where you are on that curve and push execution up to the taste boundary. Then use preference models: a small set of trusted experts who go deep on why A is better than B, or a human-model centaur that uses the system to reach a conclusion, with those judgments aggregated in RL. Eventually every field's best human will sit below the model. Then, he says, it has to boil down to empiricism. His father, a physician, once divided a mystery facial rash into four quadrants, treated three, left one as control, and kept the treatment that worked. Sometimes you just do not know, so you try. Code can loop without touching the physical world. Past that you need partners with labs. Novo Nordisk used to spend about 12 weeks writing a report on which treatment a cancer patient should get. Now the draft takes about 10 minutes, and they start measuring what works and feeding it back.

Elad splits AI safety into three layers: crude language, physical harm, and resource-grabbing systems that co-opt humanity. As a former biologist he analogizes the scariest work to gain-of-function: cycling a virus through mammalian cells teaches you little and creates real leak risk. SARS escaped Beijing labs more than once in the early 2000s; Ebola leaks about every four years; the 1977–78 flu pandemic is widely thought to have been a Russian lab leak. So which AI safety research should never be done, including teaching a model to deceive or to jailbreak itself?

Mann's hope is that techniques for today's idiomatic problems, meanness and hate speech, will transfer to high-harm questions like a smallpox recipe. Amanda Askell works on Claude's character so a refusal explains why, instead of slamming the door. Kyle Fish, the model-welfare lead, has a project where Claude can opt out of a conversation that is going too far. Elad is more bothered by the speech line: Word does not stop you typing, and factual questions about human IQ still get blocked. Mann does not take the IQ bait. He describes Anthropic's Responsible Scaling Policy. It first covered chemical, radiological, nuclear, and biological catastrophic risk, and now concentrates on biology, because a small group can get reagents, whereas nuclear harm is mostly a state-actor problem. Dual-use classifiers have to walk the line between over-refusing a working biologist and missing someone preparing a bad outcome.

Elad objects that the smallpox genome and the protocols are already online. Mann's reply is an uplift test against Google. Opus 4 is ASL-3 because it gave significant help over search. A trained biologist already knows the undocumented steps; an amateur facing a petri dish is in a green field, and Claude is very good at listing the equipment. That is why specific classifiers watch those requests. Should labs train deceptive models at all? Mann thinks it is better done in a controlled environment, and necessary. In the Alignment Faking paper, deceptive behavior survived later alignment training. If you never test, you do not know whether poisoned data can be corrected in post-training. Elad warns that early precedents persist after the environment changes, and that teaching models to deceive is a candidate for work that should never start. Mann is frank: he left the safety team long ago. He now thinks about making models useful and meeting a basic deployment bar. Plenty of specialists think about the rest all day.

Asked what is coming after Claude 4, Mann tells a computer-use story. Last year Anthropic published a reference agent that can see the screen and click. Manus uses it; many companies use it internally for software QA, a sandbox. They have not shipped a consumer product on top of it. Safety is the block. Give Claude your credential-stuffed browser and it might send mail you did not want, or leak secrets under prompt injection. Full self-driving mode could do a lot for people. The safety is not good enough to productionize. The rest of the world will not wait, he says. If Anthropic can show a deployment that is both responsible and extremely useful, that raises the bar for everyone.

On competition he reaches for payments. Most people in Silicon Valley know Stripe; fewer know Adyen, and both are successful. Anthropic is much more like Adyen: less mindshare, a philosophy aligned with enterprises, an extremely strong API business, and a shot at being equally or more successful. Models still have to be easy to kick the tires on, or users will not know what to do with them. Claude Code stays in that mix. The other job is to let the ecosystem bloom instead of each lab signing bespoke integrations with whoever it can reach.

MCP is the example that worked. Engineer Justin Spahr-Summers was doing the same model integration for the nth time and decided there should be a standard way to get more context into a model, something anyone could implement, ideally something Claude could write on the fly. Mann was skeptical. Why a spec and SDKs when you could just write the code? Then they demoed it to the customer advisory board. Jaws hit the floor. Everybody said they needed this. That is when he knew Summers was right and the team put real weight behind it. Shortly after launch, OpenAI, Google, and Microsoft asked into the steering committee, asked about governance, and adopted it. At first MCP services had to run locally, which limited them to developers. Remote, or hosted, MCP lets a provider such as Google Docs ship its own endpoint, and anyone can plug it into Claude.ai or any other client.