Gemini 3.1 Pro: Benchmarks, Animated SVGs, and CAD Demos

Matthew Berman reviews Google’s February 2026 model release, comparing reported benchmark results and examining examples of generated animations, an urban-planning app, and CAD code.

MBMatthew Berman연사 · Forward Future
채널
Matthew Berman
게시일
2026년 2월 20일
영상
6:22
읽기
4분
슬라이드
6
원문
영어
YouTube에서 열기
스크롤하면 영상이 오른쪽 아래로 작아져 계속 재생됩니다. 자막은 읽기 언어를 따릅니다.
읽기 언어2개 언어로 읽을 수 있음
이 글은 아직 한국어 번역이 없어 영어 원문으로 표시합니다.
요약

30초 요약

Matthew Berman surveys Google’s Gemini 3.1 Pro launch, highlighting reported gains in reasoning and coding benchmarks and examples of visual and interactive generation. He is enthusiastic about the demos but says benchmark scores tell only part of the story and wants to try the model in his own workflow.

  1. 1A playful visual benchmarkBerman compares animated animals generated as SVGs and jokingly calls the improvement AGI; the examples are illustrations, not a test of general intelligence.
  2. 2Tool conditions matterThe Google table shows 44.4% on Humanity’s Last Exam without tools and 51.4% with search and code under the stated setup.
  3. 3A jump on ARC-AGI-2Google reports 77.1% for Gemini 3.1 Pro versus 31.1% for Gemini 3 Pro in the displayed comparison. The ARC Prize verified score applies to this benchmark, not a general measure of AGI.
  4. 4Benchmarks are incompleteBerman wants to see how the model feels in actual use, including his OpenClaw workflow.
  5. 5A technical drawing becomes CAD codeA demonstration shows code and a rendered object generated from a drawing; the video does not independently validate the model’s dimensions or printability.

핵심 정리

  1. Google reported 77.1% on ARC-AGI-2 for Gemini 3.1 Pro, more than double the Gemini 3 Pro score in its comparison.
  2. The reported Humanity’s Last Exam score differs substantially depending on whether search and code tools are available.
  3. Berman argues that benchmark results need to be complemented by practical experience.
  4. Google’s curated examples include animated SVGs, an interactive flocking simulation, an urban-planning app, and code for a CAD model.
  5. A generated CAD example does not establish dimensional accuracy or manufacturing readiness.
전문

챕터별로 다듬은 전문 읽기

원본 자막만 보기
01

From pelicans to a new model

Berman opens with animal animations and introduces Gemini 3.1 Pro. He connects the release to the recently updated Deep Think mode.

It is Gemini day: Google has released Gemini 3.1 Pro. I want to start with what I jokingly call Pelican Bench: an animated SVG of a pelican riding a bicycle. The newer example looks more detailed and fluid than the Gemini 3 Pro version. Google also shows a frog on a penny-farthing, a giraffe in a tiny car, and an ostrich on roller skates. My AGI remark is a joke; these are illustrative visual examples, not a measure of general intelligence.

A pelican on a bicycle
슬라이드 1 / 6A pelican on a bicycleGoogle’s side-by-side SVG examples contrast Gemini 3 Pro with 3.1 Pro.

Google also said the Deep Think upgrade announced the previous week used the newer Gemini 3.1 core, and is bringing 3.1 Pro to consumer and developer products. I am eager to try it in OpenClaw. The launch figures that follow are Google’s reported comparisons from February 2026, so their settings and the date matter.

02

Reading the benchmark table

He compares Google’s reported scores for academic reasoning, ARC-AGI-2, science, coding and tool use. He also discusses Sonnet 4.6 and the limits of benchmark comparisons.

Google’s reported benchmark comparison
슬라이드 2 / 6Google’s reported benchmark comparisonThe table distinguishes tool settings and lists results across academic reasoning, coding and agentic tasks.

On Humanity’s Last Exam, Google reports 44.4% for Gemini 3.1 Pro without tools and 51.4% with code and search subject to a blocklist. Both improve on the displayed Gemini 3 Pro figures. The table places it ahead of Opus 4.6 without tools but a little behind it with tools; the two numbers within each column are not directly interchangeable because the setups differ.

ARC-AGI-2 tests abstract visual reasoning with examples of a pattern and its solution, then asks the model to solve a new instance. Google reports an ARC Prize verified 77.1% for Gemini 3.1 Pro, compared with 31.1% for Gemini 3 Pro and 68.8% for Opus 4.6 in the table. It is a striking result on that test, though a benchmark score does not establish that a system has achieved AGI.

An ARC-AGI-2 example
슬라이드 3 / 6An ARC-AGI-2 exampleThe screen illustrates the kind of visual pattern transformation discussed in the benchmark section.

For GPQA Diamond, the table reports 94.3% with no tools. On SWE-bench Verified, Gemini 3.1 Pro gets 80.6% in a single-attempt setup, near the listed 80.8% for Opus 4.6 and ahead of the older Gemini 3 Pro result. These are task-specific comparisons under the configurations shown, not guarantees of performance on every scientific or coding task.

The other interesting model in the table is Anthropic’s Sonnet 4.6. In my recent experience it has been a workhorse for knowledge work, with a personality I like, strong tool calling and a million-token context window. At the time of this video I find it expensive, although less so than Opus, which is why I am not using it as my primary OpenClaw model. That is my personal assessment and the pricing context may change.

Google also lists strong results on LiveCodeBench and SciCode, and 99.3% on the telecom portion of τ2-bench, which measures agentic tool use. But benchmarks only tell part of the story. What matters in practice is how a model behaves when you use it. I briefly switched to Gemini 3 Pro in OpenClaw after my Anthropic OAuth access was revoked, liked it, and want to see how 3.1 performs.

03

Visual and interactive demos

The examples move from a chameleon SVG and flocking simulation to a city-planning application. Berman then examines code that generates a CAD object from a technical drawing.

Google describes 3.1 Pro as intended for problems requiring more than a simple answer. Its chameleon SVG comparison shows a more detailed design and eyes intended to track the cursor. I am not sure why the launch puts such emphasis on SVGs, but the visual difference in the examples is striking. These are selected demonstrations and individual results can vary.

Chameleon SVG comparison
슬라이드 4 / 6Chameleon SVG comparisonThe selected examples show a simple earlier design and a more detailed animated chameleon.

Another example is a simulation of starling murmuration: a flock moving together in shifting patterns. Google presents the interactive simulation as something built with the model. It illustrates a creative coding application, though this demonstration alone does not establish that it faithfully models real bird behavior.

Google DeepMind chief scientist Jeff Dean also shared an urban-planning application created with Gemini 3.1 Pro. In the demonstration, it designs a city around a landscape and offers settings for paths, collisions, water and land features. It builds city blocks and produces an image suggesting what the result could look like. I find the interface impressive, even if a demo is not evidence that its plans are suitable for a real city.

Urban-planning app demonstration
슬라이드 5 / 6Urban-planning app demonstrationJeff Dean’s post introduces an illustrative city-design app created with Gemini 3.1 Pro.

The final example catches my attention because I have been getting into 3D printing. A prompt asks for ManifoldCAD code to recreate an object from a technical drawing, and the demonstration shows the generated code and a rendered three-dimensional part. I imagine using this approach for many designs; the example does not verify every measurement, however, so a person would still need to inspect dimensions, geometry and manufacturing suitability before printing or using a part.

CAD object generated from a technical drawing
슬라이드 6 / 6CAD object generated from a technical drawingThe demo displays generated ManifoldCAD code beside a rendered part; accuracy is not independently verified here.

같은 채널의 다른 글

슬라이드 12장34:21EN
Matthew Berman· 2026년 10월 27일Matthew Berman Examines Anthropic’s IPO Filing, AI Growth and GovernanceMatthew Berman, Sam Altman · Forward FutureThe presenter reviews the S-1’s financial metrics, compute strategy and voting structure while questioning the tension between rapid AI development and safety.AnthropicAI Safety엔터프라이즈 AI+2
0:26EN
Matthew Berman· 2026년 10월 25일Sam Altman on Human Values and Problem-Solving in an AI WorldSam Altman · OpenAIIn a short interview excerpt, Altman considers the enjoyment of solving hard problems and the human role in interpreting results.Human-AI CollaborationFuture of WorkAI Strategy
슬라이드 2장0:36EN
Matthew Berman· 2026년 10월 25일Kimi K3.1 and a Rubik’s Cube Solver DemonstrationMatthew Berman · Forward FutureMatthew Berman shows a scrambled virtual cube being solved and gives the Kimi K3.1 test a favorable assessment.평가(Evals)코딩 에이전트Open-weight models
슬라이드 2장0:21EN
Matthew Berman· 2026년 10월 24일An AI-Generated Kimi Benchmark Chart Used Invented ScoresMatthew Berman · Forward FutureMatthew Berman asks for a graph of Kimi K3.1 Pro benchmarks, then discovers that the displayed figures are illustrative placeholders.Benchmarks평가(Evals)Generative media
슬라이드 6장25:50EN
Matthew Berman· 2026년 10월 24일Kimi K3.1 Pro Tested: Strong 3D Demos, Uneven Results and Competitive PricingMatthew Berman · Forward FutureMatthew Berman tests the open-weight model on coding, 3D rendering and visual explanations, then compares benchmark results and prices.Open-weight modelsBenchmarksAI cost & performance+1
슬라이드 2장32:20EN
Matthew Berman· 2026년 10월 23일Who Is Local AI For? The Costs, Trade-offs and a Hybrid FutureMatthew Berman · Forward FutureMatthew Berman weighs data control and independence against hardware cost, setup complexity and inference speed.Local inferenceOpen-weight modelsAI cost & performance+1