Kimi K3: Coding Benchmarks, Cost and the Open-Weight AI Race

Matthew Berman examines Kimi K3’s early benchmark results, its cost per task and a Rubik’s Cube simulator, while weighing their limits.

MBMatthew BermanModeration · Forward Future
Kanal
Matthew Berman
Veröffentlicht
18. Juli 2026
Video
12:12
Lesezeit
5 Min.
Folien
7
Original
Englisch
Auf YouTube öffnen
Beim Scrollen schrumpft das Video in die rechte untere Ecke und läuft weiter. Untertitel folgen Ihrer Lesesprache.
LesespracheIn 2 Sprachen verfügbar
Diesen Artikel gibt es noch nicht auf Deutsch, daher wird das Original (Englisch) angezeigt.
ZUSAMMENFASSUNG

In 30 Sekunden

In July 2026, Matthew Berman reviews Moonshot AI’s Kimi K3, a large mixture-of-experts model with strong early results on selected coding and writing evaluations. He discusses inference cost, speed, the competition between open and proprietary models, and his own small application demo. His geopolitical forecasts and training-data allegations are separated from established results.

  1. 1Frontend benchmarkThe displayed Arena chart gives Kimi K3 a 76% win rate and Claude Fable 5 63% in that comparison.
  2. 2Price per task mattersBerman compares DeepSWE task success with average cost and cautions that token pricing alone is insufficient.
  3. 3Evidence has limitsBenchmarks need production testing, and Anthropic’s allegation does not establish how K3 was trained.
  4. 4Cube simulatorBerman reports that his generated application scrambles and solves the displayed cube after a long build.

Das Wichtigste

  1. Kimi K3 led the frontend Arena chart shown in the video, a result limited to that comparison.
  2. The model has 2.8 trillion total parameters and a one-million-token context window, according to Moonshot.
  3. Cost per completed task depends on token use and agent behavior as well as token prices.
  4. At the time of the video, the displayed official page said full weights were planned for release by July 27, 2026.
  5. A single working Rubik’s Cube demo is illustrative, not a broad reliability evaluation.
VOLLTEXT

Den bearbeiteten Text nach Kapiteln lesen

Nur das Transkript
01

Kimi K3 and frontend coding

Berman introduces K3 and a frontend comparison. He discusses its size and a promotional demonstration.

Moonshot AI's Kimi K3 has attracted attention for strong results against leading proprietary models on some tests. I think this may be an important moment for open-weight AI, but the story is bigger than a single leaderboard: it includes deployment cost, speed and competition between US and Chinese labs. I've also asked the model to build a Rubik's Cube simulator, which I'll try at the end.

The Arena frontend development chart shown in the video places Kimi K3 at a 76% win rate, ahead of Claude Fable 5 at 63% and GPT-5.6 Sol at 58%. That is an impressive result on this particular comparison, but it does not establish that K3 is the best model for every kind of frontend work, much less every task.

Frontend Code Arena comparison
FOLIE 1 / 7Frontend Code Arena comparisonThe displayed chart gives Kimi K3 a 76% win rate in this frontend comparison.
Kimi K3 documentation at release
FOLIE 2 / 7Kimi K3 documentation at releaseThe page describes the model and states a planned date for full weights.

Moonshot describes K3 as a 2.8-trillion-parameter model with a one-million-token context window, intended for long-horizon coding, knowledge work and reasoning. It is a very large model, unlikely to fit on a typical home computer. Its mixture-of-experts design activates only a fraction of its total parameters per token, and size alone does not determine quality.

I have talked before about the economics of US open models. I compare K3's scale and reported capability with Thinking Machines' 975-billion-parameter open model, while acknowledging that parameter count is not everything. An open-weight model's availability is distinct from publishing all training data and everything needed to reproduce it.

In a short promotional demo, Moonshot presents a simulated game world with a character, real-time reflections and a changing daylight cycle. The company says K3 edited the demo video itself. The design and 3D assets look impressive, though a selected promotional example is not an independent measurement of general performance.

02

Pricing and benchmark economics

He compares token pricing and the DeepSWE task-cost chart. Results depend on the particular models and setup.

The pricing displayed in the video is $0.30 per million cached input tokens, $3 per million uncached input tokens and $15 per million output tokens. I compare those rates with GPT-5.6 Sol and estimate that K3 is roughly half the price per token in the comparison. Those are the displayed rates at the time, not a guarantee of current prices or total cost.

Kimi K3 token pricing
FOLIE 3 / 7Kimi K3 token pricingThe video shows separate cached and uncached input rates and an output rate.

Per-token price is only part of the calculation: if a model consumes more tokens or takes more steps to finish a job, a cheaper token can still produce a similar bill. The DeepSWE chart plots task success against average task cost; the desired direction is higher success and lower cost. Its snapshot covers a particular set of coding tasks and agent configurations.

DeepSWE cost and success
FOLIE 4 / 7DeepSWE cost and successThe chart compares average task cost and completion rates for specific agent configurations.

In the chart I show, Kimi K3 Max and GPT-5.6 Sol are around $4.70 per task, with K3 a little lower in completion rate. I infer that K3 uses roughly twice the tokens for the same work given my price comparison, but the chart alone does not establish that exact token ratio: input, output, caching and agent behavior can all affect cost. GPT-5.6 Sol Max scores higher on this chart at greater expense.

03

The broader model race

Berman discusses US and Chinese competition, other benchmark claims, and the limits of public evidence. He distinguishes his forecasts from demonstrated facts.

US AI adviser David Sacks calls K3's frontend ranking and other scores concerning. I think the model is competitive on some measures, while Fable and GPT-5.6 appear more generalized across other work. These are benchmark-specific observations rather than proof of a single, permanent ranking of national capabilities.

My view is that a patchwork of US regulation can slow frontier labs, while Chinese labs continue to release strong models. I cite delayed or withdrawn releases as part of that argument, but the reasons for individual release decisions are not established by the leaderboard. I also want to be clear that I appreciate Chinese labs making capable models available openly.

Next.js evaluation claim
FOLIE 5 / 7Next.js evaluation claimThe displayed post reports benchmark results and cautions that benchmarks are incomplete.

Vercel CEO Guillermo Rauch reports that K3 leads the Next.js web engineering evaluation, with a comparable success rate to Fable in less time. The screenshot notes a top success rate of 92%, rising to 96% with assistance; those figures apply to that evaluation and its setup. A separate early internal writing benchmark cited in the video ranks K3 first for its editorial voice at 2,840 Elo, at an estimated $0.25 per script.

Internal writing evaluation
FOLIE 6 / 7Internal writing evaluationA posted early result ranks Kimi K3 for a particular editorial voice.

We need caveats: benchmarks can become saturated, and real production use matters. Anthropic has accused Moonshot and other labs of unauthorized large-scale extraction of Claude outputs for model distillation; that is Anthropic's allegation, not an established finding about the source of K3's performance or its training data. Open weights and technical descriptions also do not by themselves disclose every training input or make the model fully reproducible.

For much enterprise work, I don't think a user necessarily needs the absolute frontier model. A strong open-weight alternative can nevertheless put pressure on proprietary providers and raise competition. At the time of this July 18, 2026 video, the Kimi page shown on screen said full model weights were planned for release by July 27, so the availability claim needs that timing qualification.

I also caution against assuming a public leaderboard reveals the full state of competition. Closed labs may have newer models in testing before they launch; my estimate that Anthropic could be eight to ten months ahead, and my speculation about unreleased Fable or GPT models, are opinions rather than verified information about their internal systems.

04

Open models and the cube test

He considers possible economic benefits and hardware dependencies. His generated Rubik’s Cube simulator completes and appears to scramble and solve.

I welcome open releases because researchers, developers and companies can build on shared advances, and competitive pressure can improve products. I expect lower costs to encourage more usage, creating opportunities for applications, inference providers and chipmakers. That is an economic argument about possible effects, not a demonstrated outcome of this one model release.

The risk I see is a potential dependency if US enterprises build on Chinese models that later become optimized for Chinese chips. That is a scenario I have discussed elsewhere, not something this model launch establishes. Meanwhile, my own cube experiment has taken more than 30 minutes, which fits a criticism I've heard that K3 can be slow and token-hungry.

Rubik’s Cube simulator
FOLIE 7 / 7Rubik’s Cube simulatorThe generated application displays the cube during a scramble.

The Rubik's Cube simulator finally finishes. The cube has reflections, the scramble appears to work, and when I press solve it returns to a solved state. It's an encouraging example of building an interactive application, though a single visual check cannot establish reliability across all cube states or general coding performance.

Mehr aus diesem Kanal

12 Folien34:21EN
Matthew Berman· 27. Okt. 2026Matthew Berman Examines Anthropic’s IPO Filing, AI Growth and GovernanceMatthew Berman, Sam Altman · Forward FutureThe presenter reviews the S-1’s financial metrics, compute strategy and voting structure while questioning the tension between rapid AI development and safety.AnthropicAI SafetyEnterprise-KI+2
0:26EN
Matthew Berman· 25. Okt. 2026Sam Altman on Human Values and Problem-Solving in an AI WorldSam Altman · OpenAIIn a short interview excerpt, Altman considers the enjoyment of solving hard problems and the human role in interpreting results.Human-AI CollaborationFuture of WorkAI Strategy
2 Folien0:36EN
Matthew Berman· 25. Okt. 2026Kimi K3.1 and a Rubik’s Cube Solver DemonstrationMatthew Berman · Forward FutureMatthew Berman shows a scrambled virtual cube being solved and gives the Kimi K3.1 test a favorable assessment.EvalsCoding-AgentenOpen-weight models
2 Folien0:21EN
Matthew Berman· 24. Okt. 2026An AI-Generated Kimi Benchmark Chart Used Invented ScoresMatthew Berman · Forward FutureMatthew Berman asks for a graph of Kimi K3.1 Pro benchmarks, then discovers that the displayed figures are illustrative placeholders.BenchmarksEvalsGenerative media
6 Folien25:50EN
Matthew Berman· 24. Okt. 2026Kimi K3.1 Pro Tested: Strong 3D Demos, Uneven Results and Competitive PricingMatthew Berman · Forward FutureMatthew Berman tests the open-weight model on coding, 3D rendering and visual explanations, then compares benchmark results and prices.Open-weight modelsBenchmarksAI cost & performance+1
2 Folien32:20EN
Matthew Berman· 23. Okt. 2026Who Is Local AI For? The Costs, Trade-offs and a Hybrid FutureMatthew Berman · Forward FutureMatthew Berman weighs data control and independence against hardware cost, setup complexity and inference speed.Local inferenceOpen-weight modelsAI cost & performance+1