Theo Tests Meta’s Muse Code: Fast, Cheap and Uneven

Muse Spark 1.2 handles rapid analysis and pull-request triage at low cost, but Theo’s experiments expose integration failures, hallucinations and rate limits.

TBTheo BrownePonente · Ping Labs
Canal
Theo - t3․gg
Publicado
7 ago 2026
Vídeo
44:42
Lectura
9 min
Diapositivas
7
Original
inglés
Abrir en YouTube
Al desplazarte, el vídeo se reduce a la esquina inferior derecha y sigue reproduciéndose. Los subtítulos siguen tu idioma de lectura.
Idioma de lecturaDisponible en 2 idiomas
Este artículo aún no está en español, así que se muestra el original en inglés.
RESUMEN

En 30 segundos

Theo Browne tests Meta’s Muse Code beta and Muse Spark 1.2, comparing benchmark claims and pricing with hands-on coding, design and repository analysis. He finds the model promising for fast, low-cost triage while cautioning against trusting it with complex changes without review.

  1. 1A substantial pricing trade-offContributor pricing is much lower than standard pricing, with a different data-use policy. The figures shown are from the time of the video.
  2. 2Fast analysis, mixed assessmentsMuse produces a codebase investigation quickly; other models disagree about its relative quality.
  3. 3Quick game prototypes with broken controlsThe model builds two versions in minutes, but the 3D version needs repeated fixes.
  4. 4An investigation goes off courseMuse initially researches the wrong product and later fails to deliver a working T3 Code integration.
  5. 5A promising triage use caseTheo reports an audit of 222 pull requests in under five minutes at roughly ten cents on the contributor tier. He treats it as a starting point for review.

Conclusiones

  1. At launch, the contributor tier was much cheaper but allowed submitted data to be used to improve products.
  2. The model produced analyses and prototypes quickly, but some generated code and investigations were wrong.
  3. Parallel subagents encountered rate limits even after Theo switched tiers.
  4. A pull-request audit was useful for prioritization, but its recommendations still need human review.
  5. Model rankings and cost comparisons should be interpreted in light of the actual tasks and conditions.
TEXTO COMPLETO

Lee el texto editado por capítulos

Solo la transcripción
01

Meta returns to coding models

Theo introduces Muse Code, the model and Meta’s background with large codebases. He examines the company’s benchmark and product claims.

I’m a fan of many of Meta’s software contributions, from React and React Native to the Llama open-weight models. Yet the AI industry has moved far ahead of Meta, and its Muse models have mostly been confined to a web interface. Now Meta is releasing a coding-focused model and a terminal agent around it.

Muse Code is a beta terminal coding agent powered by Muse Spark 1.2. Meta’s published benchmarks don’t make it the leader across the board, but a model can still be valuable if it is fast, useful and inexpensive. I’m going to test the model and tool, look at the prices and see what quirks appear.

Meta describes an agent that plans changes, writes code and validates results across large repositories. Its experience with large codebases is relevant: Meta invested heavily in internal tools, including Mercurial-based workflows and stacked changes, and even adapted languages to make its codebases easier to work with.

Meta’s benchmark comparison
DIAPOSITIVA 1 / 7Meta’s benchmark comparisonThe charts compare Muse Spark 1.2 against other models on coding benchmarks.

The charts show Spark 1.2 making a substantial jump over 1.1. On the displayed Terminal Bench comparison it sits just behind Opus 5 and slightly above GPT-5.6 Terra; on Deep SWE it trails those two but exceeds Grok 4.5. I’m cautious about the internal benchmark and the models Meta chose to compare.

Meta says specialized background agents keep context during a session, while larger tasks can be split among subagents in isolated worktrees. It also describes a long-running kernel optimization experiment and a local event log intended to make tool calls and edits auditable and allow interrupted work to resume.

02

Pricing, benchmarks and speed

The contributor and standard tiers differ in price and data use. Theo examines outside benchmark and throughput figures.

At the time of the video, the standard tier lists $1.25 per million input tokens, $0.15 cached input and $4.25 output. The contributor tier lists $0.10, $0.002 and $0.20 respectively, and the pricing page says its data is used to improve products. That discount comes with a meaningful data-use choice.

Standard and contributor pricing
DIAPOSITIVA 2 / 7Standard and contributor pricingThe page shows per-million-token prices and the tiers’ different product-improvement data policies at the time.

An Artificial Analysis comparison puts Spark 1.2 at an intelligence index score of 54, up from earlier versions, with an estimated $0.40 per task at standard pricing. Theo estimates the contributor price could bring that task cost down substantially; that estimate depends on the actual mix of input, cached input and output.

The benchmark also reports a lower hallucination rate alongside a lower attempt rate. That is useful context, but I don’t want to mistake a benchmark for a guarantee: later in this very test, the model confidently pursues the wrong target. My direct experience matters alongside the aggregate numbers.

The model is closed-weight, and I’m disappointed to see it behind an earlier open-weight competitor despite Meta’s resources. Still, benchmark rank alone doesn’t settle whether a model is useful. My experience with other leading models has not always matched their ranking when I tried to merge their code.

Artificial Analysis shows roughly 30,000 output tokens per task for Spark in the comparison I’m reading, versus around 36,000 for Fable 5 and 17,000 for GPT-5.6 Sol. Those figures are workload-specific, but they give another dimension for comparing cost and model behavior.

OpenRouter’s measurements show high throughput: about 191 output tokens per second on average and 162 at the median for Spark, with higher peaks. Those numbers reflect particular providers and conditions, but the speed is noticeable when I use it in a coding session.

03

Repository analysis and game prototypes

Muse investigates event sourcing and writes a report, which other models compare. It rapidly creates game prototypes that have noticeable bugs.

I open Muse Code in the open-source T3 Code repository on the contributor tier and ask for an architectural overview. It returns in under 30 seconds. I follow up with a harder question about event sourcing, including platform differences, bandwidth and lag.

The CLI picks up my Claude Code skills and personal rules, which is an interesting compatibility choice. I ask it to summarize its event-sourcing findings in HTML, and it finds the relevant skill. The resulting analysis flags things such as snapshots, payload size and a global sequence coupling threads.

Event-sourcing investigation
DIAPOSITIVA 3 / 7Event-sourcing investigationMuse lists potential correctness and performance issues and loads an HTML reporting skill.

To compare, I give similar investigative work to Fable and DeepSeek V4 Flash, then ask the models to compare the findings. Muse produces feedback and an HTML page in under a minute; the other runs take longer. Speed is impressive, but the more important question is whether the findings are accurate.

Fable’s comparison credits Muse with broader coverage in some places but generally favors its own investigation for accuracy and root causes. DeepSeek takes a more favorable view of Muse. These are model judgments, not independent ground truth, so I treat the comparisons as clues rather than definitive evaluations.

I also ask the model to remake my fish game in 2D and 3D. It produces the 2D version in roughly two and a half minutes and the 3D version in under five. That turnaround is remarkable compared with some of my earlier runs, but the first 3D result has broken controls.

The 2D game has some appealing animation, shadows and contrast, even though it is rough. That visual taste feels different from the results I’ve seen from other models. The 3D version gets a quick mouse-control fix, but fish swim backwards and vertical look is still wrong.

Further fixes correct the fish direction but leave other interaction problems, including controls that overlap. The model is getting work done quickly and can make useful touch-ups; that does not mean the result is complete or reliable without trying it yourself.

04

Design and integration experiments

Theo reviews generated web designs and asks Muse to integrate itself into T3 Code. The investigation reveals hallucinations, limits and missing integration surfaces.

A generated web design
DIAPOSITIVA 4 / 7A generated web designTheo examines page layout and scrolling details among the model’s design examples.

I look at several generated designs on a comparison site. Some scrolling behavior, section layouts and small floating details are appealing, though there are familiar template-like elements, too much text and stylistic choices I dislike. Overall the designs have a distinct flavor, even when they aren’t my favorite.

For a harder task, I open an isolated worktree and ask Muse to investigate integrating Muse Code as a T3 Code provider. I want it to examine the existing codebase, available documentation, possible SDKs and an ACP-style integration, and to divide the research among subagents.

The terminal’s live voice transcription is surprisingly pleasant, and Muse quickly starts specialized subagents. Then they run into rate limits. A feature designed around parallel work is frustrating when the API limits prevent that work from progressing smoothly.

The dashboard shows roughly $0.13 spent so far, including the game rewrites and codebase audit, though I don’t know how current that meter is. The contributor tier is cheap, but it also has low rate limits and permits the submitted data to be used for product improvement.

The integration investigation goes badly at first: the agent assumes I mean Antigravity and researches the wrong thing. When challenged, it admits that it made the assumption without confirming it. I have to tell the agent that Muse is the very CLI it is running in.

Correcting the integration target
DIAPOSITIVA 5 / 7Correcting the integration targetThe agent acknowledges its mistaken Antigravity investigation and describes its findings about Muse Code.

After correcting the target, the agent reports limited integration surfaces: no open-source CLI or public SDK it can find and no obvious drop-in ACP path. I ask Fable to investigate separately and review Muse’s plan. Some of the generated page layout is broken, another reminder to inspect artifacts rather than trust their existence.

Fable provides feedback on the plan, and I pass it back to Muse, which updates the plan very quickly. I like using a second model to critique a first model’s work. Fast, low-cost models make that loop feel practical, especially for background tasks and categorization.

05

Comparing plans and testing the implementation

Another model compares two plans, and Theo encounters repeated rate limits. The attempted integration does not work.

I also have GPT-5.6 Sol compare the two independently produced integration plans. At this point the Muse-side planning meter is around $0.40, while my Fable and Opus investigation and reviews have cost nearly $32. These are my separate runs, not a controlled comparison of equivalent work.

Parallel subagents hit the rate limit again. I switch from contributor to the higher-priced tier, but still encounter limits. The harness itself feels polished and relatively minimal, yet recurring limits are especially annoying on a metered, per-token service.

A model’s comparison of integration plans
DIAPOSITIVA 6 / 7A model’s comparison of integration plansThe scorecard favors Fable’s plan over Muse’s; it is an AI assessment rather than an independent evaluation.

Sol’s comparison scores the Muse plan 4.8 out of 10 and the Fable plan 7, with Muse weaker on repository/API fidelity and end-to-end completeness. That is one model’s weighted evaluation, not an objective measurement, but it reinforces my reluctance to trust a cheap model with broad changes without review.

Who is this model for? People at Meta, people who enjoy experimenting, and those who value speed and low cost may have reasons to try it. But once I switch pricing tiers, the meter rises from about $0.40 to $5.32 during ten minutes of work, illustrating why the tier and workload both matter.

The agent produces a Muse integration for T3 Code, so I start the development server to test it. This was exploratory work with limited documentation, making the task difficult. The model adds a UI choice but does not properly add the provider in settings, and the integration does not work in my test.

That failure makes me cautious about trusting it with long, complex work. It seems good at organizing subagents and avoiding collisions, but less capable at seeing a complicated task through. For code I need to merge, I may prefer a slower model that is more likely to get the change right.

The CLI itself is one of the most impressive parts of the release: it feels stable and polished, even if it isn’t my favorite harness. Meta also offers ways to use the model through other tools. I would still like to see Muse Code itself opened up for integration.

06

Pull-request audits and final assessment

A fast PR audit suggests a useful niche for inexpensive analysis. Theo weighs that result against the model’s weaknesses on complex coding tasks.

I give it one more test: audit my open T3 Code pull requests, classify which need work or may be mergeable, and produce a prioritized list with confidence estimates. It encounters a GitHub API error but works around it, and completes the initial review in about four minutes.

Pull-request audit dashboard
DIAPOSITIVA 7 / 7Pull-request audit dashboardThe generated page presents PR links, merge status and confidence estimates for review.

The audit page includes clickable PR links, merge status and confidence scores in a layout I find easy to read. That is useful code-adjacent analysis. I would not let the model merge changes on its own, but an inexpensive overview can help me decide where to look first.

I extend the audit to open PRs updated in the past five days. The tool launches many subagents, and I expect more rate limits. Even so, this is the kind of broad, parallel categorization where speed and price may compensate for a need to verify the details afterward.

For now this is a model for enthusiasts: it produces results with a different flavor, but I would not make it my default for all coding. I find it interesting precisely because its strengths and weaknesses differ from the models I usually use. I also wish the CLI were open source.

In under five minutes the larger audit indexes and reviews 222 pull requests; I estimate it cost about $0.10 on the contributor tier. The page organizes a large amount of work into a form I can inspect. That is a promising result for triage, not proof that all 222 judgments are correct.

The low figure depends on the contributor tier and its data-use terms; Theo estimates a much higher cost on the standard tier. Even so, the result suggests applications such as PR categorization, summaries, titles and log exploration, provided a person checks important conclusions and merge decisions.

I don’t expect Spark to replace the strongest models for every task, and the test had obvious ups and downs. Still, the combination of speed, price and distinctive behavior makes it fun to experiment with. I can see myself using it to organize work and surface useful signals from a lot of noise.

Más de este canal

5 diapositivas38:47EN
Theo - t3․gg· 28 oct 2026Theo’s Long-Running Agent Experiments: Verification, Compute and CostTheo Browne · Ping LabsA skateboard game and two compiler projects show how repeated checks and flexible compute can sustain agent work—and how expensive those loops can become.Agentes de programaciónIngeniería de agentesAI cost & performance+2
5 diapositivas36:07EN
Theo - t3․gg· 27 oct 2026Theo on Token Costs, Coding Agents, and the Price of AI CapabilityTheo Browne · Ping LabsA viral estimate of Lauren's AI spending prompts a closer look at cache pricing, benchmarks, and the changing work of coding agents.AI cost & performanceAgentes de programaciónInference+1
5 diapositivas30:37EN
Theo - t3․gg· 26 oct 2026Theo’s Three Skills for Reviewing and Shipping with Coding AgentsTheo Browne · Ping LabsHow sol-loop, full-send and full-stop encode review, verification and different levels of merge authority.Agent skillsAgentes de programaciónCode review & verification+2
2 diapositivas0:43EN
Theo - t3․gg· 23 oct 2026Theo Distinguishes Security Vulnerabilities from Active AttacksTheo Browne · Ping LabsA short explanation of why disclosed software flaws and reported attacks are different kinds of security news.CybersecuritySoftware engineering
6 diapositivas29:22EN
Theo - t3․gg· 22 oct 2026Theo on Security Vulnerabilities, Stolen API Keys and AITheo Browne · Ping LabsA look at the gap between vulnerability disclosure and patching, and a speculative account of why stolen inference keys may be valuable.CybersecurityAI securitySoftware engineering+1
4 diapositivas31:13EN
Theo - t3․gg· 20 oct 2026Theo on Why Smarter Models Help More Than DevelopersTheo Browne · Ping LabsFrom game modifications to compiler experiments, Theo revisits his assumptions about who benefits from more capable AI.Agentes de programaciónDeveloper productivityHuman-AI Collaboration+1