Claude Sonnet 4.5: Coding, Long Tasks and a Simulated Desktop

Matthew Berman reviews Anthropic’s launch claims and tries a generated-interface demo.

DADario Amodei연사 · Anthropic
채널
Matthew Berman
게시일
2025년 10월 1일
영상
15:54
읽기
3분
슬라이드
3
원문
영어
YouTube에서 열기
스크롤하면 영상이 오른쪽 아래로 작아져 계속 재생됩니다. 자막은 읽기 언어를 따릅니다.
읽기 언어2개 언어로 읽을 수 있음
이 글은 아직 한국어 번역이 없어 영어 원문으로 표시합니다.
요약

30초 요약

Berman examines Claude Sonnet 4.5’s reported coding results and discusses longer agent runs. He also tries an interactive desktop simulation; the video’s 30-hour run is a different measure from METR’s task-completion horizon.

  1. 1Reported benchmarksAnthropic compares coding and other task results under specific settings.
  2. 2Task horizonMETR measures task difficulty in human time at a specified success rate.
  3. 3A different 30-hour measureOne long run does not establish the same task horizon.
  4. 4Simulated desktopThe demo generates interfaces and simulated messages.

핵심 정리

  1. Launch benchmarks describe specific vendor-reported evaluations.
  2. A 30-hour agent run is not equivalent to a 30-hour METR task horizon.
  3. The desktop demo generates simulated interfaces rather than sending real mail or browsing the web.
  4. Efficiency and cost matter as well as the duration of an agent run.
  5. Pricing and rankings in the video are historical.
전문

챕터별로 다듬은 전문 읽기

원본 자막만 보기
01

Release and benchmarks

Berman reviews the 2025 launch and company-reported evaluations.

I review Anthropic’s Claude Sonnet 4.5 launch, focusing on coding, longer agent tasks and a generated-interface demo. This 2025 video was sponsored by Vultr; model rankings, prices and availability are historical.

Anthropic presents Sonnet 4.5 as a leading coding model and reports results on SWE-bench Verified, Terminal-Bench, tool use, computer use and math. These are company-reported evaluations under particular setups, not proof that the model is best at every task.

02

Long tasks and efficiency

He discusses METR’s task-horizon chart and efficiency.

I discuss METR’s work on task horizons: it measures tasks by how long a human takes and estimates how long a task a system can complete at a specified success rate. The chart in the video uses a 50% success measure on a particular task set.

METR task-horizon chart
슬라이드 1 / 3METR task-horizon chartThe chart relates human task duration to an estimated 50% success rate.

I compare that chart with an Anthropic report of a model working for more than 30 hours. These are different measurements: the elapsed length of one agent run does not establish a 30-hour METR task horizon, and the run involves a surrounding agent system.

Beyond duration, I want systems to complete tasks efficiently. I discuss Greg Kamradt’s interest in action efficiency and the notion of intelligence per watt, while acknowledging that it is difficult to define one comparable measure across tasks.

03

Generated desktop demo

He tests simulated email, calculator, to-do and browser interfaces.

I try an interactive concept demo called Imagine with Claude, which generates desktop-style interfaces from prompts. I see it as a possible direction for software, but that is a prediction rather than evidence that current applications are obsolete.

A prompt creates a futuristic email interface with simulated messages. Clicking and replying cause the demo to generate more interface content; I acknowledge that the example is a simulation and does not actually send an email.

Generated email interface
슬라이드 2 / 3Generated email interfaceThe demo creates a simulated desktop and email view.

I ask for a calculator and a to-do list, then interact with the generated views. The demo illustrates dynamically produced interfaces, although the latency and underlying implementation limit what can be inferred from a few interactions.

I ask for a browser and simulated searches. The demo generates representations of Google, Wikipedia and Reddit rather than actually browsing those sites, so its pages should not be mistaken for live web results.

04

Reactions and claims

He discusses reported evaluations, safety claims and historical pricing.

I mention a reported jailbreak without reproducing the unsafe material, and show a playful website demo. A single reported jailbreak or playful output does not by itself characterize the model’s full safety or coding performance.

Box’s reported evaluation
슬라이드 3 / 3Box’s reported evaluationThe post describes a document extraction test and reported improvement.

Box’s Aaron Levie reports testing 40,000 fields across more than 1,500 documents and finding a four-point overall improvement over Sonnet 4. This is a company-reported result for that evaluation, not an independent guarantee for other datasets.

I discuss a purportedly extracted system prompt and speculate about why it contains certain instructions. The video does not independently authenticate the full prompt, and my explanation of its authors’ motivations is conjecture.

Anthropic CEO Dario Amodei says Claude writes much of the code supporting Claude and future versions. I connect that claim to longer autonomous workflows; the video does not independently measure the proportion of code or prove an unrestricted self-improvement loop.

I read launch pricing and encourage viewers to try the model. Those figures describe the release period, and present-day costs and model availability may differ.

같은 채널의 다른 글

슬라이드 12장34:21EN
Matthew Berman· 2026년 10월 27일Matthew Berman Examines Anthropic’s IPO Filing, AI Growth and GovernanceMatthew Berman, Sam Altman · Forward FutureThe presenter reviews the S-1’s financial metrics, compute strategy and voting structure while questioning the tension between rapid AI development and safety.AnthropicAI Safety엔터프라이즈 AI+2
슬라이드 2장0:36EN
Matthew Berman· 2026년 10월 25일Kimi K3.1 and a Rubik’s Cube Solver DemonstrationMatthew Berman · Forward FutureMatthew Berman shows a scrambled virtual cube being solved and gives the Kimi K3.1 test a favorable assessment.평가(Evals)코딩 에이전트Open-weight models
슬라이드 6장25:50EN
Matthew Berman· 2026년 10월 24일Kimi K3.1 Pro Tested: Strong 3D Demos, Uneven Results and Competitive PricingMatthew Berman · Forward FutureMatthew Berman tests the open-weight model on coding, 3D rendering and visual explanations, then compares benchmark results and prices.Open-weight modelsBenchmarksAI cost & performance+1
슬라이드 2장32:20EN
Matthew Berman· 2026년 10월 23일Who Is Local AI For? The Costs, Trade-offs and a Hybrid FutureMatthew Berman · Forward FutureMatthew Berman weighs data control and independence against hardware cost, setup complexity and inference speed.Local inferenceOpen-weight modelsAI cost & performance+1
슬라이드 6장1:03:37EN
Matthew Berman· 2026년 10월 23일Matthew Berman’s Team Tests Kimi K3.1 on Animation, Games, and ChartsMatthew Berman, Alex, Brian · Forward FutureA live team AMA turns into an informal test of Kimi K3.1, from personal-agent workflows to motion graphics, a browser game, and a benchmark chart.코딩 에이전트Generative media평가(Evals)+2
0:53EN
Matthew Berman· 2026년 10월 22일Sam Altman on AI and the Discovery of New KnowledgeSam Altman · OpenAIAltman expects progress in mathematical discovery to extend to biology, materials and energy.AI for scienceMathematicsComputational Biology+1