Theo finds a security use for Gemini 4 Carbon despite coding frustrations

Theo received early access to Google’s Gemini 4 Carbon and initially found it disappointing compared with the improvement he wanted over Gemini 4 Argon. After further testing, however, a security-related coding task gave him a result he had not expected.

He notes that Carbon appeared near the top of a public model ranking when announced, before newer competing models arrived. In his view, benchmark scores and the models selected for a chart do not on their own determine whether a model is the right tool for a particular job.

Google describes Carbon as a model for software engineering, long-running agent work, scientific reasoning and cybersecurity. Theo says he initially paid too little attention to the security claim, even as the model’s score on a coding benchmark seemed much better than his own experience.

His tests focus on analyzing existing codebases and reviewing proposed changes. In a Rust architecture review, he counted 23 false claims among 79 statements from Carbon, compared with 24 among 36 for Argon. He says several competing models made far fewer false claims on the same task.

For a separate T3 Code analysis, he counted 23 false claims among 86 for Carbon, versus 47 among 92 for Argon. Across his evaluations he says the share of false claims fell from 36% to 18%: a major improvement in his tests, but still too high for him to trust without checking.

He also complains about elapsed task time. Citing a third-party test, he says Carbon averaged about 22 minutes on low effort and 27 minutes on high, while an Opus comparison ranged from about three to 25 minutes. In his view, output tokens per second alone do not capture the wait a user experiences.

Theo reads an introductory API price of $2 per million input tokens and $10 per million output tokens, with cached reads at $0.10 per million. He says the listed prices double on December 1. He questions the value at either price and urges viewers to account for the scheduled change.

He says his own API tests also found inconsistent cache hits, reporting misses after five- and 30-second intervals. Because a miss can increase the cost of repeating context, he argues for comparing cost per completed task rather than relying solely on the advertised token price.

Google highlights knowledge-work and data-analysis capabilities as well as a multimodal benchmark developed with Artificial Analysis. Theo then turns to interfaces and coding, where his assessment is based on the examples he inspected rather than every possible task.

Looking at generated website designs, he finds many use similar layouts: centered two-line text, a pill-shaped element and a cropped product screenshot. He sees some variation but thinks the results become repetitive, whether or not he applies a design skill.

He contrasts Google’s claims about complex coding work with his own difficulty trusting long code analysis. He says he had reverted changes made by the preceding Argon model, and wants to know how other teams obtained better results. Google’s examples also include 3D scenes.

In a 3D test involving fish and water, the first result does not impress him. A request to make it beautiful improves the water but breaks other elements, so he asks Carbon itself to repair the scene instead of letting a different model fix its mistakes.

Theo says his early-access status prevents him from reliably measuring how much Carbon usage a paid Gemini subscription permits. He plans to revisit the question when he can obtain ordinary usage figures, while noting his past experience with subscription limits.

He summarizes Google’s claims about cybersecurity benchmarks, reported open-source findings and improvements on some safety evaluations. These are claims he is discussing from the release, distinct from the security work he later tried himself.

For codebase analysis and review, he says Carbon still invents database column names, line numbers and architectural details in some sessions. Those failures make it hard for him to trust a result even when the model displays strengths elsewhere.

Theo mainly uses Gemini through T3 Code. He says subscription access in other tools should be used only where Google permits it, and describes working with the DeepMind team on bindings that let a tool control the authorized interface.

Some hallucinations he encountered came from an older integration that did not send the model all the information it needed, including line numbers. He credits DeepMind engineers with investigating and improving the connection after his feedback, while saying the end-to-end experience remains slow for him.

A security report from an independent developer changed the shape of his evaluation. The reporter had used an Anthropic security program to audit T3 Code and identified two issues; Theo says fixes had been released by the time the video went live and asks users to update.

He first worked through the report with Anthropic’s Mythos. One proposed response would have required a user to retrieve a code from the other machine before approving a connection. Theo considered that too burdensome for his remote-machine workflow and continued discussing an alternative with the model.

After several exchanges, he says they reached a solution that treated connections through T3 Code’s trusted service differently from raw links. Before the work was finished, he gave the same report to Carbon to compare its response.

Carbon described the issue in a way he understood and proposed separating known-machine flows from raw URL flows. Theo says this matched the approach he wanted and identified additional concerns. He was surprised because other models had refused the task or initially suggested a less convenient design.

He remains critical of the time and back-and-forth required to use Carbon. He asked other models to review its work, supplied feedback and stopped the process to film before it had fully finished. He considers security incident response a potentially useful application, while noting his early-access status could affect the experience.

Theo thanks the independent reporter, Brandon, for a clear and actionable report. He says the researcher asked for little in return and highlights the value of outside contributions to an open-source project’s security.

In another test, he asked Carbon to implement an investigation concerning remote updates on Windows. He says the work was progressing well and received favorable reviews from other models, though those scores reflect those particular reviews rather than a complete assessment.

Despite the security examples, Theo would not make Carbon his default coding model. In his experience Opus remains more pleasant for routine work, while Carbon can be slow, expensive and prone to mistakes or invented details in larger codebases.

He nevertheless considers Carbon a substantial improvement over Argon in staying on task and following a conversation. He plans to keep testing how useful its feedback is over time rather than treating one result as a complete verdict.

The 3D repair task eventually finishes after roughly 25 minutes. Theo finds the scene somewhat changed but still flawed: elements around the fish do not appear where he expected. Based on this example, he would choose other tools for similar 3D work.

His conclusion is that Carbon may be useful for security development and is an improvement for people already committed to Gemini, but he still prefers alternatives for much of his coding. He remains hopeful that later releases will address the problems he encountered.