Claude 3.7 Sonnet and Claude Code: An Early Coding Test

ZUSAMMENFASSUNG

In 30 Sekunden

Berman examines Claude 3.7 Sonnet’s reasoning modes and company-reported benchmarks before trying Claude Code on an evolving Snake game. His historical tests also explore math and the apparent absence of current web information, with limited evidence from a small set of examples.

1Model and coding toolClaude 3.7 Sonnet and Claude Code were separate products in the launch.
2Benchmark contextThe 70.3% SWE-bench result uses custom scaffolding.
3Two AI snakesThe host iterates from a basic game to competing players and a moving obstacle.
4Recent informationThe tested interface did not answer a current-events question in the video.

Das Wichtigste

Claude 3.7 Sonnet combined standard responses and an extended-thinking mode at launch.
Anthropic’s 70.3% SWE-bench Verified result used custom scaffolding in the displayed chart.
The generated Snake demo runs with two AI-controlled players, but is not a comprehensive software test.
Model agreement on an integral is not proof of correctness.
A launch-period web-access test does not describe later product capabilities.

ThemenCoding-Agenten Benchmarks Anthropic Reasoning & test-time compute

VOLLTEXT

Den bearbeiteten Text nach Kapiteln lesen

Transkript zeigen

Nur das Transkript

The 2025 release

Berman explains the hybrid reasoning model, coding tool and company benchmark charts. The results are specific to the displayed configurations.

In February 2025 I review Claude 3.7 Sonnet and the Claude Code research preview, then test coding and mathematical tasks. I build up a Snake game with AI-controlled players; the video is an early hands-on look rather than a controlled comparison.

Anthropic presents Claude 3.7 Sonnet as a hybrid reasoning model that can give standard responses or use extended thinking. Claude Code is a separate command-line tool for agentic coding. Product features and access described here reflect the launch period.

I describe the visible thinking display and the ability for API users to set a thinking budget. The display should not automatically be assumed to expose every internal step, and token budgets, output limits and context windows are distinct settings.

Anthropic’s SWE-bench Verified chart shows 62.3% for Claude 3.7 Sonnet and 70.3% with custom scaffolding, compared with around 49% for the listed prior configurations. These are company-reported results under particular setups; the scaffold makes the comparison more than a raw model-only test.

FOLIE 2 / 4TAU-bench tool-use comparisonThe vendor chart compares selected models on retail and airline tasks.

A separate TAU-bench chart shows the new model ahead of the displayed alternatives on retail and airline tool-use tasks. I also discuss math, science and visual reasoning benchmarks, but those selected evaluations do not guarantee performance on all workflows.

Building Snake with Claude Code

He creates a game and adds AI players and a moving obstacle. The short test shows the game running.

I decide to update my older testing rubric because newer reasoning models do well on it. I note that my earlier Grok 3 test was not as demanding as it could have been, so the informal examples here should not be read as a fully controlled model comparison.

Claude Code quickly produces a basic Snake game in my demonstration. I then ask it to add AI control; after applying the generated changes I can toggle the agent and change speed, and the snake uses an A-star pathfinding approach to reach food.

The AI-controlled snake eventually makes a mistake and loses. I ask for a second automated snake, then add multiple food items and a temporary moving block that can defeat the other snake. The displayed game runs, though a short demonstration does not validate every rule or edge case.

FOLIE 3 / 4Two AI-controlled snakesThe generated game shows two snakes and moving blocks during the host’s demonstration.

Math and recent information

He compares mathematical responses and asks about a recent event. He closes with a qualified early impression.

For a hard integral, Grok 3 and Claude give different answers. I compare Claude’s result with o3-mini and tentatively prefer the agreement, but agreement between models is not itself a proof of a mathematical result.

I try the Basel problem using extended thinking. Claude gives the familiar result that the sum of reciprocal squares is π²/6, and I ask for a step-by-step explanation. Getting a famous result does not by itself prove that the derivation is sound.

Finally I ask about a recent Apple announcement and find that the tested interface does not answer with current information. I regard the apparent lack of live web access in this 2025 test as a drawback; the video does not establish the current capabilities of later versions.

My early impression is that the model performs well in the coding examples while having limitations. The experiments are a small set of launch-period demonstrations, so broader claims about reliability need additional testing.