Claude Opus 4.5: Launch Benchmarks, Pricing and Tool Use

Matthew Berman reviews Anthropic’s November 2025 release, its coding benchmarks and developer tools, while comparing it with contemporary models.

MBMatthew BermanVortrag · Forward Future
Kanal
Matthew Berman
Veröffentlicht
24. Nov. 2025
Video
10:27
Lesezeit
3 Min.
Folien
4
Original
Englisch
Auf YouTube öffnen
Beim Scrollen schrumpft das Video in die rechte untere Ecke und läuft weiter. Untertitel folgen Ihrer Lesesprache.
LesespracheIn 2 Sprachen verfügbar
Diesen Artikel gibt es noch nicht auf Deutsch, daher wird das Original (Englisch) angezeigt.
ZUSAMMENFASSUNG

In 30 Sekunden

In a November 2025 video sponsored by Warp, Matthew Berman reviews Anthropic’s Claude Opus 4.5 launch benchmarks, historical pricing, an agent evaluation example and advanced tool-use features. He also discusses token efficiency and early user impressions. The results are specific to their reported evaluations and settings, not a universal ranking of coding or computer-use ability.

  1. 1Coding benchmarksBerman reviews Anthropic’s SWE-bench Verified results and observes that the chart’s truncated axis exaggerates visual differences.
  2. 2Launch pricingHe compares historical input and output token prices, noting that actual costs depend on usage.
  3. 3Engineering exerciseAnthropic described strong results on its take-home task; a later account specifies the comparison’s time constraints.
  4. 4An airline benchmark exampleAnthropic reported an alternate solution that the benchmark scored as a failure; its appropriateness depends on the scenario.
  5. 5Advanced tool useTool search loads relevant definitions on demand, while programmatic calls and examples address other tool-use challenges.
  6. 6Output-token efficiencyBerman compares output token counts and scores on a specific SWE-bench setup, not all tasks.

Das Wichtigste

  1. Anthropic reported 80.9% on SWE-bench Verified for Opus 4.5 without extended thinking in its launch evaluation; this is a benchmark-specific result.
  2. Launch API pricing was $5 per million input tokens and $25 per million output tokens, a historical rate subject to change.
  3. Tool search, programmatic tool calling and usage examples were designed to help agents use large tool sets more efficiently; they do not create an unlimited context window.
  4. In Anthropic’s later account of a two-hour engineering exercise, Opus 4.5 matched the best time-limited human result, not an unrestricted comparison with every engineer.
  5. The video includes a sponsored Warp promotion and early user testimonials.
VOLLTEXT

Den bearbeiteten Text nach Kapiteln lesen

Nur das Transkript
01

Launch benchmarks and pricing

Berman compares reported coding, agent and reasoning scores and historical API prices. The figures reflect specific 2025 evaluation conditions.

Anthropic has released Claude Opus 4.5, shortly after other frontier model launches. In this November 2025 video, I review the company’s reported benchmarks, prices and developer features. The video also contains a paid sponsorship for Warp.

SWE-bench Verified launch comparison
FOLIE 1 / 4SWE-bench Verified launch comparisonAnthropic’s chart shows reported scores on 500 tasks; the displayed vertical axis is truncated.

Anthropic’s launch chart reports 80.9% for Opus 4.5 on SWE-bench Verified, compared with 77.2% for Sonnet 4.5 and other displayed model scores. The vertical axis starts near 70 rather than zero, making the bars look farther apart. The difference from Sonnet is 3.7 percentage points, and the result applies to the reported 500-task setup, not all coding work.

Berman reviews other reported agent and computer-use results, including Terminal-Bench 2.0, τ²-bench and OSWorld. He also identifies tests where competitors scored higher, such as graduate-level reasoning and visual or multilingual tasks. Results reflect specific harnesses, configurations and datasets; missing competitor figures do not establish their performance.

He compares a vending-machine simulation leaderboard and ARC-AGI scores, noting that the displayed Opus result did not lead each chart. These are historical snapshots with their own scoring rules, and neither benchmark alone measures every aspect of intelligence or workplace performance.

Anthropic’s launch price was $5 per million input tokens and $25 per million output tokens. Berman compares it with the contemporary Gemini 3 Pro price tiers. A real bill depends on the input-output ratio, caching, prompt length and applicable prices, so one percentage difference is not universal.

02

Engineering assessment and sponsorship

He describes Anthropic’s performance-engineering test and presents a sponsored Warp segment. A later Anthropic account clarifies the test’s time-limited comparison.

Berman says Anthropic tested Opus 4.5 on a difficult performance-engineering hiring exercise and describes it as better than every hired candidate. In a later, more detailed account, Anthropic said a prerelease model matched the best human result under a two-hour limit; the human had used Claude with steering, and stronger human results were possible without the time limit. This is a task-specific comparison, not proof of superior engineering ability in general.

The video includes a paid promotion for Warp, an AI coding product. Berman describes its claimed benchmark placement, interface and support for parallel agents and tools. Those are promotional claims from the sponsor segment, not an independent evaluation in this video.

03

Agent evaluation and tool discovery

Berman discusses an airline-service benchmark example and Anthropic’s advanced tool-use features. The tools reduce some context overhead rather than eliminating limits.

Berman describes Anthropic’s τ²-bench example: an airline-service task expected refusal of a basic-economy change, while the model proposed upgrading the cabin first and then changing the flight. Anthropic characterized this as a legitimate alternative that the benchmark marked wrong. Berman questions whether it was actually optimal; authorization, cost and the scenario’s rules matter.

An alternate airline-service solution
FOLIE 2 / 4An alternate airline-service solutionAnthropic describes a model response that differed from the benchmark’s expected outcome.

Anthropic also introduced advanced tool-use features. Berman explains that loading definitions for many MCP tools up front can consume context before the user’s request. A tool-search mechanism can retrieve relevant definitions when needed, reducing upfront overhead; it does not give the model literally infinite tools or an unlimited context window.

He outlines three features: tool search, programmatic tool calling in a code-execution environment, and examples showing how to use a tool. Programmatic calling can keep intermediate results out of the model’s context, while examples can clarify usage that a schema alone may not convey.

Tool definitions and context overhead
FOLIE 3 / 4Tool definitions and context overheadThe example estimates the context consumed by definitions from several MCP servers.

Anthropic’s example estimates that definitions for several MCP servers can occupy tens of thousands of tokens. Berman contrasts a conventional approach with on-demand discovery and reports a large context reduction in the illustrated setup. Actual savings depend on the tools, schemas and requests.

04

Efficiency and early reactions

He reviews a specific output-token comparison and early user impressions. These are historical results and testimonials, not guarantees.

Effort and output-token comparison
FOLIE 4 / 4Effort and output-token comparisonAnthropic’s chart compares SWE-bench scores and output-token use under specified settings.

Berman then discusses an Anthropic chart comparing SWE-bench results with output-token use. He observes that Opus 4.5 achieves a higher score at fewer output tokens than Sonnet 4.5 in that example. The plotted settings are specific, and output tokens alone are not the complete cost or a universal measure of intelligence.

Finally, he cites early reactions from Dan Shipper and Ethan Mollick, who praised coding and practical work. These are user testimonials rather than controlled independent proof that the model is best for every application.

Mehr aus diesem Kanal

12 Folien34:21EN
Matthew Berman· 27. Okt. 2026Matthew Berman Examines Anthropic’s IPO Filing, AI Growth and GovernanceMatthew Berman, Sam Altman · Forward FutureThe presenter reviews the S-1’s financial metrics, compute strategy and voting structure while questioning the tension between rapid AI development and safety.AnthropicAI SafetyEnterprise-KI+2
0:26EN
Matthew Berman· 25. Okt. 2026Sam Altman on Human Values and Problem-Solving in an AI WorldSam Altman · OpenAIIn a short interview excerpt, Altman considers the enjoyment of solving hard problems and the human role in interpreting results.Human-AI CollaborationFuture of WorkAI Strategy
2 Folien0:36EN
Matthew Berman· 25. Okt. 2026Kimi K3.1 and a Rubik’s Cube Solver DemonstrationMatthew Berman · Forward FutureMatthew Berman shows a scrambled virtual cube being solved and gives the Kimi K3.1 test a favorable assessment.EvalsCoding-AgentenOpen-weight models
2 Folien0:21EN
Matthew Berman· 24. Okt. 2026An AI-Generated Kimi Benchmark Chart Used Invented ScoresMatthew Berman · Forward FutureMatthew Berman asks for a graph of Kimi K3.1 Pro benchmarks, then discovers that the displayed figures are illustrative placeholders.BenchmarksEvalsGenerative media
6 Folien25:50EN
Matthew Berman· 24. Okt. 2026Kimi K3.1 Pro Tested: Strong 3D Demos, Uneven Results and Competitive PricingMatthew Berman · Forward FutureMatthew Berman tests the open-weight model on coding, 3D rendering and visual explanations, then compares benchmark results and prices.Open-weight modelsBenchmarksAI cost & performance+1
2 Folien32:20EN
Matthew Berman· 23. Okt. 2026Who Is Local AI For? The Costs, Trade-offs and a Hybrid FutureMatthew Berman · Forward FutureMatthew Berman weighs data control and independence against hardware cost, setup complexity and inference speed.Local inferenceOpen-weight modelsAI cost & performance+1