Matthew Berman Reviews Claude Sonnet 4.6: Benchmarks, Computer Use and Safety

A launch-day overview examines Anthropic’s reported knowledge-work and agent results, along with prompt-injection and capability evaluations.

MBMatthew BermanVortrag · Forward Future
Kanal
Matthew Berman
Veröffentlicht
17. Feb. 2026
Video
11:49
Lesezeit
4 Min.
Folien
9
Original
Englisch
Auf YouTube öffnen
Beim Scrollen schrumpft das Video in die rechte untere Ecke und läuft weiter. Untertitel folgen Ihrer Lesesprache.
LesespracheIn 2 Sprachen verfügbar
Diesen Artikel gibt es noch nicht auf Deutsch, daher wird das Original (Englisch) angezeigt.
ZUSAMMENFASSUNG

In 30 Sekunden

Matthew Berman reviews Anthropic’s February 2026 launch of Claude Sonnet 4.6, including pricing, a beta one-million-token context window, computer use and benchmark comparisons. He regards it as a strong knowledge-work model and highlights cases where it scores close to or above Opus on particular tests. He also discusses prompt injection and Anthropic’s ASL-3 deployment and capability-threshold assessment. The results are model- and benchmark-specific, and the vending chart depicts a simulation.

  1. 1Computer-use improvementAnthropic’s chart reports 72.5% for Sonnet 4.6 versus 61.4% for Sonnet 4.5 on OSWorld-Verified.
  2. 2Prompt-injection exposureWeb content can contain instructions intended to redirect an agent. Anthropic reports improved resistance but the test does not guarantee immunity.
  3. 3Coding and tool benchmarksThe launch table reports gains on Terminal-Bench, SWE-bench Verified and MCP-Atlas compared with Sonnet 4.5.
  4. 4A simulated vending businessVending-Bench Arena shows a higher ending balance for Sonnet 4.6 after more than 350 simulated days.
  5. 5Safety thresholds and uncertaintyAnthropic says the model did not cross two specified thresholds but that confidently ruling out those thresholds is becoming harder.

Das Wichtigste

  1. Sonnet 4.6 launched with the same starting API prices as Sonnet 4.5 and a one-million-token context window in beta.
  2. Anthropic reported gains in coding, computer use and tool-use evaluations, with results varying by benchmark.
  3. Prompt injection remains a concern even when a model improves on resistance tests.
  4. The vending comparison shows simulated balances rather than actual operating profits.
  5. Anthropic deployed the model under ASL-3 and judged that it did not cross its AI R&D-4 or CBRN-4 thresholds, while describing uncertainty in those evaluations.
VOLLTEXT

Den bearbeiteten Text nach Kapiteln lesen

Nur das Transkript
01

A workhorse launch

Berman introduces Sonnet 4.6, its launch pricing, beta context window and stated focus. He reads the computer-use comparison.

Anthropic has introduced Claude Sonnet 4.6 as a workhorse model, with reported improvements in coding, tool use, agentic work and computer use. At launch in February 2026, it offered a one-million-token context window in beta and became the default on Claude’s Free and Pro plans. I am excited about a fast model aimed at practical knowledge work.

Launch pricing and context window
FOLIE 1 / 9Launch pricing and context windowThe launch page lists starting API prices and describes the one-million-token context window as beta.

The announced API price starts at $3 per million input tokens and $15 per million output tokens, the same starting rates as Sonnet 4.5. Anthropic emphasizes consistency and instruction following, as well as tasks such as building presentations and working with spreadsheets through products including Claude Cowork and Claude Code.

OSWorld scores over time
FOLIE 2 / 9OSWorld scores over timeThe chart compares Sonnet computer-use scores and notes the change to OSWorld-Verified.

Anthropic’s OSWorld chart shows Sonnet 4.5 at 61.4% and Sonnet 4.6 at 72.5% on OSWorld-Verified. The benchmark asks an agent to complete practical tasks in a computer environment. Earlier points on the same chart use the original OSWorld rather than the later verified version, so the entire time series is not measured identically.

02

Computer-use risks

He explains prompt injection and summarizes Anthropic’s resistance claims. He notes the consequences of giving agents access to sensitive information.

Computer use works by having a model inspect a screen and act through a virtual mouse and keyboard, rather than relying on a custom connector for every task. That makes it broadly useful, but also exposes it to prompt injection: a malicious instruction hidden in material the model reads may try to redirect its behavior.

Anthropic says its safety evaluations show substantially better resistance to prompt injection than Sonnet 4.5 and performance similar to Opus 4.6. These are evaluation results, not a guarantee that attacks cannot succeed. I think users should consider that risk when giving agents access to personal data or actions with real-world consequences.

Prompt injection discussion
FOLIE 3 / 9Prompt injection discussionAnthropic describes the risk and reports improved resistance in its evaluations.
03

Benchmarks and simulated vending

The launch table compares coding, tools, reasoning and office work. A vending simulation illustrates a particular long-running task rather than a real profit record.

Anthropic’s benchmark table reports terminal coding rising from 51.0% for Sonnet 4.5 to 59.1% for 4.6, and SWE-bench Verified from 77.2% to 79.6%. OSWorld-Verified rises from 61.4% to 72.5%. On its tool-use tests, retail improves while telecom remains close to its previous score.

Coding and tool-use benchmarks
FOLIE 4 / 9Coding and tool-use benchmarksThe table compares Sonnet and other models on specific agentic evaluations.

The table also reports MCP-Atlas scaled tool use rising from 43.8% to 61.3%. On Humanity’s Last Exam, the score moves from 17.7% to 33.2% without tools and from 33.6% to 49.0% with tools. I would have expected an even larger tool-assisted jump, but these are the results shown.

For agentic financial analysis the reported score is 63.3%, and on the GDPval-AA office-task comparison Sonnet 4.6 has an Elo of 1,633, ahead of Opus 4.6’s 1,606 in the displayed table. Those are benchmark-specific results. I interpret them as evidence that Anthropic is targeting real-world knowledge work, though Opus leads many other rows.

Office and reasoning scores
FOLIE 5 / 9Office and reasoning scoresThe lower portion of the table shows GDPval-AA Elo, ARC-AGI-2 and other results.

The chart reports ARC-AGI-2 increasing from 13.6% to 58.3%, with Opus 4.6 at 68.8%. GPQA Diamond and visual reasoning also improve, while multilingual question answering is roughly unchanged. These different tests measure different capabilities and do not imply the same improvement across every use.

Vending-Bench Arena simulates running a vending business: managing inventory, purchases and reordering to improve the balance. The shown simulation ends around $5,500 for Sonnet 4.6 versus roughly $2,000 for 4.5 after more than 350 simulated days. Anthropic attributes the newer model’s result to early capacity investment followed by a late shift toward profitability; this chart is a simulation, not a real vending-machine profit record.

Vending-Bench Arena simulation
FOLIE 6 / 9Vending-Bench Arena simulationThe chart plots simulated balances for Sonnet 4.5 and 4.6 over time.
04

Product updates and safety assessment

Berman reviews product features, the ASL-3 deployment standard, GDPval-AA and the system card’s capability thresholds. He concludes with the uncertainty around measuring advanced capabilities.

The launch also describes adaptive reasoning, API context compaction in beta, and updates to web search, code execution, memory and tool calling. Anthropic says the Claude in Excel add-in supports MCP connectors. These are launch-time product claims and availability can change.

ASL-3 deployment
FOLIE 7 / 9ASL-3 deploymentThe system card states the safety standard used for deployment.

The system card says Sonnet 4.6 was deployed under Anthropic’s AI Safety Level 3 standard. I outline Anthropic’s levels as a way to understand the safeguards and risk categories. Deployment under that standard describes the company’s risk-management framework; it does not establish that every dangerous capability is present or every risk is eliminated.

GDPval-AA, developed by Artificial Analysis using tasks from OpenAI’s GDPval dataset, covers 44 occupations across nine industries. The tasks mirror professional outputs such as documents, slides, diagrams and spreadsheets. Sonnet 4.6’s higher Elo than Opus 4.6 on this particular comparison supports my view that the model is aimed at knowledge work, but does not prove it is better on every professional task.

GDPval-AA evaluation
FOLIE 8 / 9GDPval-AA evaluationThe system card describes the task set and Elo-based comparison.

Anthropic’s system card concludes that Sonnet 4.6 does not cross its AI R&D-4 or CBRN-4 capability thresholds. The former concerns fully automating an entry-level remote Anthropic researcher’s work; the latter concerns high-consequence chemical, biological, radiological or nuclear misuse. These are Anthropic’s internal definitions and assessment.

Capability threshold uncertainty
FOLIE 9 / 9Capability threshold uncertaintyThe card states its threshold determination and explains why ruling out future thresholds is difficult.

The card also says confidently ruling out those thresholds is becoming harder, because evaluation results approach high capability levels and parts of the thresholds are difficult to measure. Anthropic says it has proactively applied mitigations associated with AI R&D-4. I find the uncertainty important: assessing capability can become harder even when a threshold has not been judged crossed.

My takeaway is that Sonnet 4.6 is positioned as a knowledge-work model, and its benchmark results narrow the apparent gap with Opus on some tasks. People online speculate about how Anthropic named the model or what it was originally intended to be, but I do not know the details. I plan to test it further.

Mehr aus diesem Kanal

12 Folien34:21EN
Matthew Berman· 27. Okt. 2026Matthew Berman Examines Anthropic’s IPO Filing, AI Growth and GovernanceMatthew Berman, Sam Altman · Forward FutureThe presenter reviews the S-1’s financial metrics, compute strategy and voting structure while questioning the tension between rapid AI development and safety.AnthropicAI SafetyEnterprise-KI+2
2 Folien0:36EN
Matthew Berman· 25. Okt. 2026Kimi K3.1 and a Rubik’s Cube Solver DemonstrationMatthew Berman · Forward FutureMatthew Berman shows a scrambled virtual cube being solved and gives the Kimi K3.1 test a favorable assessment.EvalsCoding-AgentenOpen-weight models
6 Folien25:50EN
Matthew Berman· 24. Okt. 2026Kimi K3.1 Pro Tested: Strong 3D Demos, Uneven Results and Competitive PricingMatthew Berman · Forward FutureMatthew Berman tests the open-weight model on coding, 3D rendering and visual explanations, then compares benchmark results and prices.Open-weight modelsBenchmarksAI cost & performance+1
2 Folien32:20EN
Matthew Berman· 23. Okt. 2026Who Is Local AI For? The Costs, Trade-offs and a Hybrid FutureMatthew Berman · Forward FutureMatthew Berman weighs data control and independence against hardware cost, setup complexity and inference speed.Local inferenceOpen-weight modelsAI cost & performance+1
6 Folien1:03:37EN
Matthew Berman· 23. Okt. 2026Matthew Berman’s Team Tests Kimi K3.1 on Animation, Games, and ChartsMatthew Berman, Alex, Brian · Forward FutureA live team AMA turns into an informal test of Kimi K3.1, from personal-agent workflows to motion graphics, a browser game, and a benchmark chart.Coding-AgentenGenerative mediaEvals+2
0:53EN
Matthew Berman· 22. Okt. 2026Sam Altman on AI and the Discovery of New KnowledgeSam Altman · OpenAIAltman expects progress in mathematical discovery to extend to biology, materials and energy.AI for scienceMathematicsComputational Biology+1