In 30 seconds
Theo Browne argues that open-weight AI models may struggle to catch the best closed models as compute access diverges and leading labs restrict the use of their outputs for training. In T3 Code, he says, open models made up 0.5% of turns, while an altered coding benchmark showed large drops for several models. He also worries that AI assistance will accelerate research at the best-equipped labs. New hardware access or training techniques could change his forecast.
- 1Long agent tasks reveal a gap beyond benchmark scoresBrowne says the best closed models stay coherent and coordinate subagents better than open models he has tested.
- 2Open models made up 0.5% of turns in T3 CodeBrowne reports his product’s usage figures; they describe its own audience, not all AI users.
- 3Small changes to coding tasks lowered many model scoresHe describes a test derived from 89 public coding tasks and argues that familiar questions can inflate apparent capability.
- 4Faster access to GPUs could compound the divideHis chart uses fictional GPU counts to illustrate how different acquisition rates can produce a growing gap.
- 5Agent training needs more than human-written dataBrowne argues that task histories and verifiable reinforcement learning teach models workflows that ordinary examples may miss.
- 6An external report estimated faster capability growthHe cites a highly uncertain estimate of around twice the earlier rate on well-specified tasks and distinguishes it from a prediction of autonomous takeoff.
Takeaways
- Browne expects unequal access to GPUs to make competing at the frontier increasingly difficult.
- Evaluate models on changed or private tasks, not only scores on public benchmarks.
- For long-running agent work, test whether a model stays coherent and delegates effectively.
- Browne thinks restrictions on distillation make it harder to copy useful agent behaviors.
- Employee reports of productivity gains are useful signals, but they are not controlled measurements.
Read the edited text by chapter
Why the open-model gap matters
DeepSeek convinced me that downloadable models could approach the frontier. I now think that period of near parity is receding.
I love open source: I began making content by helping maintainers explain tools I used, and I am spending heavily to make T3 Code an open-source product. That makes it painful to say why I have stopped expecting downloadable models to match the best closed models for my work.
DeepSeek's releases around the end of 2024 changed my expectations. I could download a model that felt close to a commercial model from just months before, and use it in T3 Chat quickly and cheaply. When R1 arrived soon after OpenAI's reasoning model, its results convinced me that open weights could compete at the frontier.
DeepSeek and other Chinese labs also contributed techniques for training and serving models under hardware constraints. But although current open models are much better than those from months ago, I think the gap is now widening. I hope that judgment is wrong.
The capabilities hidden by a single score
Long-running agent work and usage in T3 Code lead me to a different assessment from a simple benchmark ranking.
My three reasons are access to computing power, the difficulty of distillation, and the possibility that models will help improve their successors. Before those arguments, the first question is how to measure the difference in capability.
A chart I generated to compare releases has inconsistent numbers, so I would not treat its reported delay as reliable. In my own use, Kimi K3.1 Pro is impressive; I initially thought its weights were unavailable, then learned they had been released. That release gives me some hope, although I think major open-weight releases have become less frequent.
A capability I care about is staying coherent across long, messy tasks while splitting work among subagents. In my experience the latest closed models have improved substantially at this, while Kimi K3.1 Pro does not perform that way. On this dimension I would date the gap back further than simple benchmark comparisons suggest.
Intelligence is multidimensional. Open models can score well on things that existing benchmarks measure and still miss behaviors I find necessary for long-running work. Some closed models also fall short on those behaviors. GLM Flash stood out to me because it could follow instructions and remain coherent, but I still see a gap.
Artificial Analysis changed its index, and the new version reduced open models' scores much more on average than closed models' scores. I do not think that proves a conspiracy against open models: to me, the revised ordering matches my experience better. The size of the change does raise a question about what earlier tests rewarded.

I looked at T3 Code usage. Open models account for roughly 0.5% of turns and 0.7% of tokens in the figures I show; the share of installs that used one at least once fell from 16% to 6%. That is my product's usage, and I read it as a sign that its users are choosing closed alternatives for their work.
OpenRouter reports enormous traffic to models such as DeepSeek and GLM. I argue that those totals need context: T3 Code's overall daily token volume is also substantial, and in my view free access accounts for some open-model usage. The different audiences and products make a single usage figure a limited comparison.
What changes when public test questions change
A modified coding benchmark and my own small test raise concerns about models fitting published questions.
Another concern is whether a public coding benchmark measures general ability or familiarity with its questions. I describe an experiment by Justin Lo that takes 89 public DeepSWE tasks and changes instructions or starting code while retaining similar problems. A model that learned the old answers may stumble on the modified task.
In the results I show, Opus 5.5 did not lose score on the altered tasks, while several other closed and open models did. DeepSeek V4.1 Flash fell by about 19 points, and some others also fell sharply. I interpret this as evidence of overfitting to public evaluations; the numbers alone do not explain each model's training history.

I saw a similar pattern in a small benchmark I made about skateboard tricks: one open model got every old question right but none of the newly added ones. I suspect that teams have strong incentives to optimize for published tests because high scores attract attention and financing. That can make impressive rankings misleading about performance on changed problems.
Why access to computing power can widen the gap
I illustrate how faster access to GPUs may compound into different training capacity over time.
My first structural reason is compute. Training the strongest models requires huge amounts of computing, and I think more compute continues to improve results. Chinese labs and researchers have found ways to adapt constrained hardware, including adding memory to older GPUs, but access to the latest chips remains restricted.
To illustrate, I draw invented starting GPU counts for an American closed lab and a Chinese open lab. Both may add hardware, but if one can acquire it much faster, the difference between their resources grows over time. The values on my chart are illustrative, not an estimate of either company's inventory.
Why distillation has become harder
Agent training needs task histories and verified practice; I argue that restrictions on stronger models make generating useful data more difficult.
The second issue is distillation: using outputs from one model to help train another. I have opposed strict bans partly because leading labs collected web data before much of the web became harder for later entrants to access. But I think the type of data now needed to improve agents is changing.
I argue that training after the initial pretraining phase increasingly matters, particularly reinforcement learning on tasks whose results can be checked. An agent can discover workflows unlike a human's, such as combining several tool calls into a single short script. Reading more human-written material alone does not necessarily teach that behavior.
In the workflow I describe, an agent reviews real task histories, finds opportunities to combine actions, and creates synthetic examples for further training. A lab without comparable compute or usage histories may want outputs from a stronger model to build such examples. I think restricting access to those outputs makes catching up more difficult.
I believe Anthropic has made it harder to extract useful training data from its models. Based on my own logs, I suspect the form of some reasoning traces is deliberate, although that is my interpretation. I also point to an inference provider's report about attacks and stolen credentials, which the provider connects to a market for discounted model access.
My interpretation is that some operators want the resulting interaction histories as training data as well as revenue from reselling access. I connect this to attacks targeting API keys.
Those two forces reinforce each other in my account: unequal access to compute expands the gap, while restrictions on distillation remove one way to narrow it. I think this helps explain the slower arrival of stronger downloadable models, though I acknowledge there can be other explanations and future methods.
I compare interface designs generated by Kimi and OpenAI models and see striking similarities. I take that as a possible sign of distillation, but visual similarity alone is not a definitive account of how a model was trained. I also describe a claimed restriction on reading some agent-to-agent prompts as another example of protecting capabilities.
AI-assisted research may compound the difference
An external report and employee surveys inform my concern about acceleration, though the evidence and estimates have limits.
My third factor is recursive self-improvement. I do not think we have reached a fully autonomous takeoff in which models independently make themselves smarter. I do think AI assistance may already speed up the work of making better models, and that speed matters for labs without the same tools.
I discuss an external evaluation of acceleration at Anthropic. The evaluators said much of the evidence they found most informative was redacted, making the reader more dependent on their judgment. Their questions included whether capability growth is accelerating, whether AI contributes, and what could block future growth.
The report estimated that, as of October 2026, capability growth on well-specified tasks was likely around twice the earlier pace, with substantial uncertainty and a plausible range of roughly 1.3 to three times. I explain the distinction between continuing on a prior trend and accelerating beyond it.

A chart I highlight reports how much Anthropic employees felt AI sped up their own work. Their reported multiplier rises across several model versions and reaches roughly twelve times for an internal model in the chart. These are employees' self-reports, not a controlled measurement of realized productivity.
I think access to internal research capabilities could compound the advantage. In my account, behaviors that a provider does not expose to customers are harder for another lab to reproduce through distillation. It remains possible that general coding skill or different research techniques could produce similar results.
What could change the prediction
I combine the three pressures and describe the advances that might allow open models to close the gap.
My conclusion combines all three claims: unequal compute access, harder distillation and AI-assisted research could each make it harder for open-weight models to keep up. If they all matter together, I expect the gap to increase rather than shrink.
I compare Anthropic and OpenAI as a hypothetical test of whether AI-assisted research creates an advantage, and I believe the former is moving faster on the capabilities I care about. That comparison motivates my concern about labs with fewer resources.
More hardware access, different distillation methods or new training techniques could prove me wrong. I would welcome that: downloadable models are fun, useful for ownership and understanding, and important to the ecosystem. For now, I do not expect them to match the closed models I use for demanding work.
More from this channel
32:29EN
28:16EN
45:45EN
27:57EN
42:19EN
19:39EN