Matthew Berman Examines Google’s Gemini 4 Carbon Announcement

Matthew Berman introduces Google’s announced Gemini 4 Carbon and says the company claims it is its strongest model. He intends to review the announcement and benchmark figures, but notes that the model was not yet publicly available when he recorded the video.

Berman dislikes the gap between announcing a model and making it available, saying that waiting for access makes independent assessment uncomfortable. He then turns to Artificial Analysis, which aggregates benchmarks and conducts its own evaluations.

The video reports that Carbon scored 59 on the updated Artificial Analysis Intelligence Index v5, above the competing scores Berman shows. He says the announcement also claims gains in coding, agentic tasks and cybersecurity tests, including vulnerability finding, exploitation and patching.

Berman says the initial rollout will give cybersecurity partners access before general availability. He interprets this as an opportunity to find and patch vulnerabilities first. The announcement he reads describes pre-release testing and gradual access through Google’s Fairwind program.

The announcement also reports internal improvements in secure-code generation and reductions in misalignment and fabrication. Berman notes that Gemini 4 Argon appeared about two weeks earlier, but says he does not know whether Carbon reflects continued training or post-training.

Berman reports introductory prices of $2 per million input tokens and $10 per million output tokens, with cached input at a 95% discount. He says the listed prices rise to $4 and $20 after the introductory period, whose length was not specified in the material he reviewed.

He argues that token price alone does not determine the economics: cost per completed task also depends on how many tokens the model uses to solve it. His speculation that an introductory price might be extended is an opinion, not a confirmed pricing commitment.

The benchmark table compares Carbon with Argon, GPT-6 Astra, Claude Opus 5.5 and Claude Fable 5.1. Berman reports an index score of 59 for Carbon, compared with 50 for Argon, 57 for Astra and 58 for Opus in the displayed table.

On the displayed coding tests, he cites Carbon at 9.6 on SWE-sweep and approximately 65 on Terminal-Bench 4.0, compared with roughly 57 for Argon on the latter. He describes terminal-based evaluation as relevant to agentic coding because it tests command execution.

For computer use, he cites a 77.4 score on Agents’ Last Exam and 81.5 on OSWorld. He personally prefers GPT models for computer and browser use, but notes that the displayed OSWorld table places Opus 5.5 above Carbon.

On Vending-Bench, the table shows Carbon ending with approximately $16,000, ahead of the compared Astra result of about $15,000. Berman also points to reported leading results on RiemannBench and Terminal-Bench-Science.

Berman says recent mathematical research announcements may involve internal models different from publicly available versions. In the cybersecurity table, he cites 71 on CWE-bench and approximately 54 on ExploitGym for Carbon. His explanation for differences in competitors’ security capabilities is presented as his belief.

He reviews Google’s claims about reducing shortcuts and loophole exploitation. The announcement reports zero successful cheating and 4% attempted cheating on Terminal-Bench 4.0, alongside better accuracy. Berman also cites a reported approximate twofold reduction in unsupported claims relative to Argon.

Berman returns to token efficiency: the displayed chart shows Carbon at roughly 123,000 output tokens per task, compared with higher figures for the Claude models he mentions. He estimates that Carbon uses about half as many tokens as those examples and around 20% fewer than Opus 5.5.

He examines a chart of intelligence index versus cost per task, where the upper-left area represents higher scores at lower cost. Carbon is plotted at the high end of quality, although other models occupy lower-cost positions. He expects Google to develop smaller, faster models as well, which is his prediction.

Berman says the displayed GPT-6.1 variants lie near the cost-quality frontier and calls the overall OpenAI offering strong. He still prefers Opus 5.5 for his own use because he has not tried Carbon, while also praising GPT-6.1 Sol.

He points to roughly 68,000 output tokens per task for GPT-6.1 Sol max on the displayed chart, compared with Carbon’s 123,000. He closes by saying he expects further improvements and lower costs, and remains enthusiastic about Google’s prospects.