Claude Opus 5.5: Benchmarks, Cost and Anthropic’s Approach to Pacing

Matthew Berman reviews the release and speaks with Anthropic’s Thariq about daily workflows, evaluation and agent harnesses.

ThThariqGuest · Anthropic
Channel
Matthew Berman
Published
Sep 23, 2026
Video
31:59
Read
5 min
Slides
4
Original
English
Open on YouTube
Scroll to read and the video shrinks to the bottom-right corner. Captions follow your reading language.
Reading languageAvailable in 2 languages
You're reading the original English.
SUMMARY

In 30 seconds

Matthew Berman examines Anthropic’s reported Opus 5.5 benchmarks, token pricing, effort levels and safety claims. Anthropic technical staff member Thariq joins to discuss model selection, AI-assisted development, pacing, evaluation and changes to the Claude Code harness.

  1. 1Reported coding resultsAnthropic’s table shows a Terminal-Bench increase alongside strong FrontierCode and CursorBench results.
  2. 2Price and task costBerman compares token prices and explains why total cost depends on usage and effort.
  3. 3AI helping build AIThariq describes Claude’s contributions as many small improvements with people in the loop.
  4. 4Pacing and deploymentThariq distinguishes work on unreleased frontier capabilities from making existing capabilities broadly available.
  5. 5Adapting the harnessThe Claude Code team has shortened its system prompt and introduced plugin evaluations for newer models.

Takeaways

  1. Anthropic reports strong coding and knowledge-work scores and lower typical task costs for Opus 5.5; benchmark conditions matter.
  2. Token price and cost per completed task are different measures, and more reasoning effort does not always yield a better benchmark result.
  3. Thariq describes AI-assisted development as many incremental contributions rather than a sudden fully autonomous process.
  4. He distinguishes pacing the newest frontier models from making established capabilities more accessible, while presenting Anthropic’s deployment view.
  5. Agent harnesses, tools and plugins need to adapt as model capabilities change.
FULL TEXT

Read the edited text by chapter

Transcript only
01

Benchmarks and pricing

Berman reviews Anthropic’s reported coding, knowledge-work and computer-use scores. He compares token prices, speed and cost per task.

Claude Opus 5.5 is Anthropic’s first major release after its call to pace the frontier. I see it as an important model for coding and knowledge work. Later in the video, Anthropic technical staff member Thariq joins me to discuss how the model fits into the company’s lineup and how its tools are changing.

Anthropic says Opus 5.5 performs at Fable 5.1’s level on most tasks and costs less on typical workloads than Opus 5. Berman finds its reported combination of speed, price and benchmark performance striking, especially following the company’s public discussion of pacing frontier development.

Anthropic benchmark table
SLIDE 1 / 4Anthropic benchmark tableThe release compares coding, knowledge-work and computer-use results under stated settings.

In Anthropic’s table, Opus 5.5 scores 66.4% on Terminal-Bench 4.0, compared with 57.9% for GPT-6 Astra and 55.8% for Fable 5.1. It also leads the displayed FrontierCode and CursorBench comparisons. Berman speculates that it may be smaller or distilled, but explicitly says he does not know.

On GDPval-AA, a knowledge-work evaluation, the displayed score is 1846 Elo versus 1735 for Fable 5.1 and 1542 for Astra. Berman emphasizes the jump because the benchmark covers tasks such as documents, presentations and data work. These are company-reported comparisons under specified evaluation conditions.

Opus does not lead every test in the table: Astra scores higher on AutomationBench and Terminal-Bench Science. The displayed OSWorld partial score is 81.8%, slightly above Fable’s 80.7%. Berman notes that his team increasingly uses computer control in creative tools such as video editors and Unreal Engine.

Opus API pricing
SLIDE 2 / 4Opus API pricingThe table compares input, output and cache rates for Opus 5.5 and Opus 5.

The quoted Opus 5.5 prices are $4 per million input tokens and $20 per million output tokens, versus $5 and $25 for Opus 5. Cache reads are quoted at $0.20 versus $0.50, and cache writes at $5 versus $6.25. Anthropic also says output is more than 30% faster than the prior model.

Berman argues that token price alone is not enough: a model that spends many more tokens on a task may cost more despite a low unit price. He examines pass rate or quality against cost per task, looking for combinations that deliver better results at lower total cost.

AutomationBench cost by effort
SLIDE 3 / 4AutomationBench cost by effortThe chart plots pass rate against estimated task cost for several models and effort levels.

In the displayed FrontierCode chart, medium effort for Opus 5.5 scores higher than max effort while costing much less per task. Berman uses this to argue that higher reasoning effort is not automatically the best choice for every workload. Users should test settings against their own tasks.

GDPval quality and cost
SLIDE 4 / 4GDPval quality and costThe plotted points compare Elo and estimated cost per task across effort levels.

The GDPval and Terminal-Bench cost charts also show strong results for Opus 5.5 at several effort levels. Berman highlights medium or high as potentially attractive points in the displayed Terminal-Bench comparison. These are benchmark-specific estimates, not universal costs for every task.

02

Communication, safety and efficiency

He discusses shorter model responses and company safety claims. Anthropic’s typical-workload savings estimate differs from the token-price reduction.

Anthropic says the model communicates more naturally, puts important information first and follows writing instructions. Berman values shorter explanations when switching among many coding agents, though he wants more testing before judging its creative writing against other models.

Berman reviews Anthropic’s safety claims: the company reports strong automated behavioral-audit results and broader testing on long or impossible tasks. It says the model is deployed with safeguards comparable to Fable 5.1 in biology and cybersecurity. Berman distinguishes the company’s claims from his own speculation about earlier incidents.

Anthropic estimates that typical workloads at default settings cost 40% less than with Opus 5. Berman explains that this differs from the approximately 20% reduction in listed input and output token prices: total task cost also depends on how many tokens are used. The 40% figure is the company’s workload estimate.

Berman looks at an Artificial Analysis composite index and reports that Opus 5.5 leads the displayed chart. He sees that as evidence of a large advance, while the composite still reflects the evaluations and settings included in it.

03

Thariq on models and pacing

Thariq explains how he selects models, describes AI-assisted development and discusses evaluation and pacing.

Thariq joins and describes Opus 5.5 as a good daily driver that is both cheaper and more capable. He says Fable 5.1 remains useful for planning, brainstorming, review or security work where the cost of being wrong is high. For his own website, he recently used Opus 5.5 throughout.

Asked how AI contributes to building AI, Thariq says Claude helps build Claude, including by writing code. He describes this as many small improvements accumulating over time rather than a sudden moment when a model performs all of its own development. That is his account of the workflow, not a claim of autonomous self-improvement without people.

On efficiency and quality, Thariq describes a pattern of developing frontier capability and then making it more accessible and economical. He expects this pattern to recur: a capability that first appears in an expensive model can become available more broadly over time.

Berman asks whether the ‘5.5’ name signals an inflection like earlier half-version releases. Thariq cautions against reading too much into version numbers, while saying the team is very excited about this model. He recalls earlier Sonnet and Opus releases as important coding milestones.

On pacing, Thariq distinguishes evaluating the newest, often unreleased frontier models from making established capabilities accessible at lower cost. He says difficult evaluations, including sandbox escapes, require work, while Anthropic believes its classifiers and fallbacks for the released models are robust. This is Anthropic’s position on its own deployment.

Berman asks whether the interval between internal access and public release is growing. Thariq says adding evaluators takes time, but he cannot say how the total interval will change. For him, the key is that the company and third-party evaluators consider a model safe to release.

04

Evaluation and agent harnesses

The conversation turns to visual design, remaining limitations and how Claude Code tools and plugins adapt to newer models.

Thariq says the model improved small visual-design details on his blog and did so quickly enough to keep him in the flow. Flashy games and 3D demos are impressive, but he expects responsiveness, efficiency and tool use to matter in ordinary software engineering too.

Both speakers find evaluation increasingly difficult. Thariq says a model may reason toward a correct answer yet fail to commit to it in an evaluation. He wants to help users obtain better outcomes from available intelligence, especially as more people gain access to frontier-level capabilities.

Asked what still needs work, Thariq points to computer and browser use, and to helping models express their measured capability reliably. He also wants systems to use context and memory to offer useful pushback at the right moment without becoming intrusive.

On agent harnesses, Thariq says the underlying model keeps changing, so the Claude Code team must adapt. The team recently removed much of its system prompt and introduced plugin evaluations to help authors update skills and plugins without unnecessarily constraining newer models.

Thariq is excited about better tool calling and cloud-managed agents, but says product teams still have work to do to use the model’s intelligence effectively. He describes adding some tools, removing others and continuing to adjust the harness. Berman closes by saying he has more testing to do.

More from this channel

6 slides25:50EN
MBMatthew Berman· Oct 24, 2026Kimi K3.1 Pro Tested: Strong 3D Demos, Uneven Results and Competitive PricingMatthew Berman · Forward FutureMatthew Berman tests the open-weight model on coding, 3D rendering and visual explanations, then compares benchmark results and prices.Open-weight modelsBenchmarksAI cost & performance+1
2 slides32:20EN
MBMatthew Berman· Oct 23, 2026Who Is Local AI For? The Costs, Trade-offs and a Hybrid FutureMatthew Berman · Forward FutureMatthew Berman weighs data control and independence against hardware cost, setup complexity and inference speed.Local inferenceOpen-weight modelsAI cost & performance+1
45:08EN
MBMatthew Berman· Oct 21, 2026Sam Altman on AI Discovery, Consciousness, Compute and a Human-Centered FutureMatthew Berman, Sam Altman · Forward FutureIn a conversation with Matthew Berman, Altman discusses scientific progress, personal assistants, recursive self-improvement and the importance of human agency.AI for sciencePersonal agentsConsciousness+2
5 slides12:14EN
MBMatthew Berman· Oct 14, 2026Gemini 4 Argon Tested: 3D Demos, a Spreadsheet Prototype and Benchmark ResultsMatthew Berman · Forward FutureMatthew Berman tests Google’s model on games and visual projects, then compares reported coding and agent benchmarks.GeminiBenchmarksCoding agents+1
5 slides10:36EN
MBMatthew Berman· Oct 13, 2026Meta Muse Wonder Tested: Strong Benchmarks, Mixed Demos and Quota FrictionMatthew Berman · Forward FutureMatthew Berman tests Meta’s models on browser demos and reviews their published benchmark scores, availability and cost.Models & trainingBenchmarksAI cost & performance+1
2 slides17:37EN
MBMatthew Berman· Oct 10, 2026AI Coding, Game Mashups and the Value of Human TasteMatthew Berman · Forward FutureMatthew Berman explains decompilation and game mashups, then argues that iterative AI coding changes software and creativity.Coding agentsSoftware engineeringCreativity+1