En 30 secondes
In an August 2025 video, Matthew Berman reviews Anthropic’s Claude Opus 4.1 announcement and its selected benchmark comparison. He sees gains in several coding tests but notes a lower airline tool-use score and says practical testing matters.
- 1Coding resultsThe displayed SWE-bench Verified result rose from 72.5% to 74.5%, a two-percentage-point increase.
- 2Mixed tool-use scoresTerminal-Bench increased, but the airline portion of TAU-bench fell from 59.6% to 56.0%.
- 3Compare by taskThe table shows other models ahead on GPQA Diamond and AIME 2025; Berman plans practical testing.
À retenir
- Anthropic reported 74.5% on SWE-bench Verified for Opus 4.1 versus 72.5% for Opus 4 in its displayed comparison.
- Terminal-Bench rose in the table, while the airline portion of TAU-bench declined.
- Other models led the displayed GPQA and AIME comparisons; results depend on evaluation settings and task.
- Berman treats real-world usage as an important complement to benchmarks.
Lire le texte remanié, chapitre par chapitre
The Opus 4.1 release
Berman introduces Anthropic’s update and its stated goals.
Anthropic announced Claude Opus 4.1 in August 2025 as an update to Opus 4 for agentic work, coding and reasoning. I review the company’s reported benchmark table, which reflects its selected tests and settings at the time.
Benchmarks and real-world use
He reviews coding, tool-use and reasoning scores and emphasizes practical evaluation.
On SWE-bench Verified, the displayed scores rise from 72.5% for Opus 4 to 74.5% for Opus 4.1: an increase of two percentage points. Anthropic also describes improvements in research and analysis, including tracking details and agentic search.
The table shows Terminal-Bench increasing from 39.2% to 43.3% and GPQA Diamond from 79.6% to 80.9%. TAU-bench retail rises from 81.4% to 82.4%, while airline falls from 59.6% to 56.0%. Other displayed categories show smaller gains.
In the displayed comparison, Opus 4.1 leads the included o3 and Gemini 2.5 Pro results on the two coding benchmarks, while those models score higher on GPQA Diamond and AIME 2025. These selected numbers do not establish a universal ranking.
I think the important question is how the model works on actual tasks, and I plan to test it. My view that Claude was particularly strong for agentic coding is an assessment from 2025, not a current leaderboard or a guarantee for every workflow.
Sur la même chaîne
34:21EN
Matthew Berman· 27 oct. 2026Matthew Berman Examines Anthropic’s IPO Filing, AI Growth and GovernanceMatthew Berman, Sam Altman · Forward FutureThe presenter reviews the S-1’s financial metrics, compute strategy and voting structure while questioning the tension between rapid AI development and safety.
0:26ENMatthew Berman· 25 oct. 2026Sam Altman on Human Values and Problem-Solving in an AI WorldSam Altman · OpenAIIn a short interview excerpt, Altman considers the enjoyment of solving hard problems and the human role in interpreting results.
0:36ENMatthew Berman· 25 oct. 2026Kimi K3.1 and a Rubik’s Cube Solver DemonstrationMatthew Berman · Forward FutureMatthew Berman shows a scrambled virtual cube being solved and gives the Kimi K3.1 test a favorable assessment.
0:21ENMatthew Berman· 24 oct. 2026An AI-Generated Kimi Benchmark Chart Used Invented ScoresMatthew Berman · Forward FutureMatthew Berman asks for a graph of Kimi K3.1 Pro benchmarks, then discovers that the displayed figures are illustrative placeholders.
25:50ENMatthew Berman· 24 oct. 2026Kimi K3.1 Pro Tested: Strong 3D Demos, Uneven Results and Competitive PricingMatthew Berman · Forward FutureMatthew Berman tests the open-weight model on coding, 3D rendering and visual explanations, then compares benchmark results and prices.
32:20ENMatthew Berman· 23 oct. 2026Who Is Local AI For? The Costs, Trade-offs and a Hybrid FutureMatthew Berman · Forward FutureMatthew Berman weighs data control and independence against hardware cost, setup complexity and inference speed.