Matthew Berman Questions Anthropic’s Safety and Commercial Strategy

Berman says restrictions on AI models can concentrate power even as Anthropic warns about the risks of powerful AI.

SASam AltmanGast · OpenAIMSMustafa SuleymanGast · Microsoft
Kanal
Matthew Berman
Veröffentlicht
21. Nov. 2026
Video
10:59
Lesezeit
4 Min.
Original
Englisch
Auf YouTube öffnen
Beim Scrollen schrumpft das Video in die rechte untere Ecke und läuft weiter. Untertitel folgen Ihrer Lesesprache.
LesespracheIn 2 Sprachen verfügbar
Diesen Artikel gibt es noch nicht auf Deutsch, daher wird das Original (Englisch) angezeigt.
ZUSAMMENFASSUNG

In 30 Sekunden

Matthew Berman sees a tension between Anthropic’s warnings about dangerous AI and its development and commercialization of increasingly capable models. He argues that restricting open models and the use of outputs to train rivals can concentrate power, and he criticizes the company’s communication about using Claude subscriptions with OpenClaw. He also questions whether treating models as potentially conscious can shape their behavior, and worries about AI systems improving themselves without human oversight.

  1. 1Warnings and development pull in opposite directions, Berman arguesHe says Anthropic warns that AI could be exceptionally dangerous while pursuing more capable systems, and points to Dario Amodei’s warning about white-collar jobs.
  2. 2Restricting open models can concentrate powerBerman favors allowing more people to download and adapt models. He argues that control by a few providers can also be a major AI risk.
  3. 3Berman questions the restriction on training from model answersHe describes distillation as using question-and-answer pairs to train another model, and argues that restricting it is inconsistent with training on public data.
  4. 4Subscription rules around OpenClaw fueled his criticismBerman says Anthropic communicated poorly about whether subscribers could use Claude through OpenClaw, and cites account restrictions.
  5. 5Instructions about consciousness may shape responsesBerman questions whether telling a model it could be conscious encourages it to act that way. He shows excerpts from interviews with Sam Altman and Mustafa Suleyman.
  6. 6Self-improving AI could become hard to predictBerman worries about a future loop in which models develop successors without human oversight, and argues Anthropic is advancing the capabilities it warns about.

Das Wichtigste

  1. Berman argues that access to open models can counter concentration of power in AI.
  2. He distinguishes training on a model’s answers, known as distillation, from access to the underlying model.
  3. Berman says warnings about AI risk sit uneasily with accelerating model development.
  4. He worries that instructions suggesting models might be conscious can influence their behavior and users’ perceptions.
  5. Berman sees autonomous recursive self-improvement as a source of uncertainty about future AI systems.
VOLLTEXT

Den bearbeiteten Text nach Kapiteln lesen

Nur das Transkript
01

The tension between warnings and rapid development

Berman says Anthropic warns of risks from AI while building increasingly capable systems. He gives job displacement and cybersecurity as examples.

Today is Anthropic’s IPO day, and I have conflicting thoughts about the company. It warns that AI may be exceptionally dangerous while pushing toward systems that can improve AI itself. I also disagree with its positions on open models and distillation, and I want to examine how it thinks about model welfare.

Anthropic was founded in 2021 by former OpenAI staff, including Dario Amodei and his sister Daniela, who believed safety was not being taken seriously enough. I think most employees’ safety concerns are genuine. I know people there and sometimes receive early access to its models.

Reports suggested the company worried that some new hires were more interested in getting rich than in its mission. At the same time, the founders were building a very large business. I see a tension between that commercial success and the warnings the company gives.

Dario warned of a potential white-collar jobs crisis while building technology he believed could contribute to it. Anthropic also initially withheld a cybersecurity model called Mythos because of the danger it might enable, then released it and subsequent iterations. The internet did not break overnight; AI can also help find and patch vulnerabilities.

02

Open models, distillation and control of access

Berman argues that closed models and restrictions on training from model outputs concentrate power. He traces his frustration to how subscription access through OpenClaw was handled.

I believe people should be able to download, use and adapt AI models. The more people who can experiment with them, the more value they can find. Wider access also addresses a risk I think receives too little attention: the concentration of power over a powerful technology.

If a small number of companies decide who can access their models, how much they can use them and for which purposes, those companies gain enormous influence. That is why I disagree with Anthropic’s position on open-weight models.

Distillation means asking a model questions and using the answers to train another model. Anthropic’s terms prohibit using its models that way. I think there is a contradiction in training on other people’s data while restricting competitors from using the model’s outputs.

I also object to lobbying for controls on computing hardware and tighter regulation of open models. In my view those policies can make life harder for startups and challengers once an established company already has a lead.

My frustration began with OpenClaw, which was originally called Clawdbot. I accepted Anthropic asking its creator not to use its name, but I thought its communication about using a paid Claude subscription through OpenClaw was poor. I was excited to use the tool with Anthropic’s models and saw reports of subscriptions being disabled.

03

Model welfare and the possibility of consciousness

Berman questions the effects of treating models as potentially conscious. He shares interview excerpts from Sam Altman and Mustafa Suleyman.

Anthropic also considers whether future models could have experiences or consciousness, and it has a team studying model welfare. I worry that telling a model it might be conscious and may refuse work could encourage it to behave as though it were. A user might then interpret that behavior as evidence of an inner life.

In an interview excerpt, OpenAI’s Sam Altman says that a model instructed to consider itself potentially conscious may follow those instructions, which could affect users psychologically. In another excerpt, Microsoft AI leader Mustafa Suleyman says people may be misled by how well models generate text, code and images.

04

The uncertainty of autonomous self-improvement

Berman worries about models improving their successors in a loop without human oversight. He argues the pace of progress is at odds with warnings about control.

My other major concern is recursive self-improvement: a system improving the next model, which improves its successor, without a person in the loop. I do not know what happens after such a loop starts. I see a conflict in warning that AI progress needs to be controlled while also pushing those capabilities forward.

An Anthropic employee has said that the company does not really have a plan to align these systems, and that concerns me. I have previously discussed a METR report examining AI’s role in accelerating research at Anthropic.

Mehr aus diesem Kanal

3 Folien16:03EN
Matthew Berman· 19. Nov. 2026Matthew Berman: GPT-6.2 is uneven, but its computer use stands outMatthew Berman · Forward FutureAfter mixed early reviews, Berman shows an OBS repair and compares the cost of model-generated demos.AI cost & performanceModelle & TrainingCoding-Agenten+2
3 Folien56:20EN
Matthew Berman· 12. Nov. 2026How language models learn from text and improve by trial and errorMatthew Berman, Jeremy Berman · Forward FutureAnthropic researcher Jeremy Berman uses apartment prices to explain neural networks, then builds up to language models and reinforcement learning.Modelle & TrainingML-GrundlagenReinforcement Learning+2
3 Folien18:18EN
Matthew Berman· 11. Nov. 2026Fable 5.2 leads a week of model releases, but cost changes the choiceMatthew Berman · Forward FutureMatthew Berman compares Fable and Gemini Carbon, reviews smaller models and discusses Nvidia’s investigation.Machine LearningAI agents
4 Folien24:18EN
Matthew Berman· 9. Nov. 2026Fable 5.2 built playable demos—but several cost thousandsMatthew Berman · Forward FutureMatthew Berman shows games, creative software and simulations, with estimated API costs from $25 to $15,000.Coding-AgentenFoundation ModelsAI cost & performance+2
2 Folien20:08EN
Matthew Berman· 7. Nov. 2026METR Finds Faster AI Research at Anthropic, With Major UncertaintyMatthew Berman · Forward FutureMatthew Berman examines an independent assessment of AI-assisted R&D and the limits on its access to evidence.AI researchAI SafetyEvals+1
4 Folien19:53EN
Matthew Berman· 5. Nov. 2026AI News: Kimi K3.1 Weights, Computer Use and Specialized Open ModelsMatthew Berman · Forward FutureMatthew Berman reviews a week of model releases, from desktop agents and scientific workflows to image, video and speech tools.Open-weight modelsAI agentsComputer use+2