Theo revises his Gemini 4 Carbon assessment after measuring coding delays

In my previous video about Gemini 4 Carbon, I was more positive than I intended. I valued the fact that it appeared less restrictive for some security work, but after reviewing my own usage I think I understated how slow it was in my coding workflow.

In the comparable work I audited, I found Carbon consistently about five times slower than the other models I used. That is an observation from my own tasks rather than a controlled universal benchmark. I say the difference mattered enough to change my recommendation.

Price adds to my concern: I say the model’s price is scheduled to double on December 1. I also want to distinguish the model’s capabilities from the experience of using the service that hosts it. People at DeepMind have been receptive to feedback, and I understand that they are working on the speed issue.

A chart I used earlier measured the time after a request was accepted by the server until a response completed. Many of its tasks were not multi-turn agent workflows. That view missed delays which recur when a coding agent makes several model requests and tool calls.

I examined nine review rounds in which multiple models received the same brief. The display shows Carbon completing four rounds, with an average of 16 minutes 44 seconds, compared with 3 minutes 31 seconds for Opus and 3 minutes 6 seconds for GPT-6.1 Sol on those four. In the other five, the parent agent stopped waiting for Carbon.

I had modified my agent instructions to stop waiting for Carbon when there was other work to do. That means the incomplete runs do not tell us how long those reviews would have taken to finish. I have not waited for all of them to complete, so I cannot precisely measure the full difference.

Output speed and tool-call behavior can contribute to delay, but I think the biggest problem in my sample occurs before a response begins. In the illustrated coding thread, Carbon took about 11 seconds to start a response on average. Repeating that wait across many steps can substantially lengthen an agent task.

I demonstrate one step in real time and compare it with other models. The example is a single response and its tool calls, not a complete coding task. I also say Carbon needed more steps for some of my work, which compounds the repeated startup delay.

I have seen hosting latency problems before: I describe a period when Azure-hosted OpenAI models sometimes took much longer than usual to produce a first token. I say that issue was later fixed, and I want Google’s teams to have enough information to investigate this one.

In a discussion with a Google developer, I suggested reducing overhead in repeated requests, comparing it with a persistent connection used in another coding tool. The reply pointed out that processing a long prompt before generating output—prefill—also costs time. I agree that changing the network transport alone might not solve it.

After looking more closely, I say Carbon sometimes spent longer waiting to start than another model spent completing its response. I also observed repeated reads of the same files. Both behaviors can add to total task time, although the individual causes and their contribution require further investigation.

I raise a separate prompt-cache concern. In a test shown on screen for Gemini 4 Argon, the cache fully matched 167 of 228 turns, while the other displayed models matched all 228. The display estimates higher cost when the cache misses. This is a result from that workload, not a measured miss rate for every Gemini request.

A Google representative responded that their workloads did not show the same behavior. I say I am willing to share prompts, environments and run records so the teams can reproduce and fix the problems. Another response acknowledges a latency–cost tradeoff and says the service should improve.

I also discuss differences users have reported between subscription access and hosted API endpoints, as well as integration friction when reasoning effort appears in the model identifier. These reports need to be assessed on the specific endpoint and workload; they do not isolate one cause of the latency I observed.

My revised conclusion is that, for my current coding workflow, the combined delay, cost and reliability concerns outweigh the benefits I found. I want more capable competitors and say I would welcome improvements, but I will not use this model for routine work in its current state.