Is a Faster Model Actually Faster? Measure the Finished Job, Not the Tokens

Morgan Suo has been at MiniMax for almost three years. When the company first stepped into the American market almost no one knew it, though it already had its first video generation model — something she promises to show at the end. First, though, she wants to talk about MiniMax's latest language model, due out the following week, and about the hidden cost of a faster model.

Picture an AI application built into your own workflow. A new model arrives and someone asks whether you want to try it. It finishes the task faster, so the question becomes: should you switch? You run a research task, and the report comes back very quickly — but it misses a very important source. Someone now has to find that source and fix the report, which is what happens every day. So did the job really get faster?

The talk looks at how MiniMax reduced the model's computation, and then at how to test whether those changes are worth it. To answer that, Morgan uses one concrete job.

MiniMax builds several multimodal models and products, and a quick chat reply has different needs from a research report: the task defines what good performance means. Her focus is the new large language model, M3.1 Flash, and one research job. The analyst's question: as demand for AI servers grows, are the companies supplying printed circuit boards — the boards that connect the electronic components inside devices such as AI servers — turning that growth into cash?

MiniMax's internal review describes the work across 14 company reports, and the model has several jobs. It has to find the right reports, line up the same reporting periods, keep different businesses separate, and then explain what the numbers mean.

Think about the analyst who receives the report. They need to open the source behind an important claim, check a calculation, and see what evidence could change the conclusion. A useful report gives them something they can check — and that is MiniMax's standard for a finished research job.

The review highlighted why those checks matter: profit can grow while cash flow tells a different story. A company sells more and its reported profit rises, but customers have not paid yet, or it has bought materials that are still sitting in storage. The model should point the analyst to the cash flow statement and the relevant disclosures. A report earns trust when its claims can be checked, so the sources and calculations still have to be verified, and the time and cost measured on this same job. Separate technical tests help explain specific trade-offs.

With the job clear, Morgan turns to what changed inside M3.1 Flash and ties each change to a simple question. Where can attention work be reduced? How is data stored and moved? How does the serving setup affect the time a user waits? And how much reasoning does the model do? These choices touch different parts of the job, and each needs a check against the result you want: every optimization has to help the finished job.

Start with long inputs. Every model now advertises a huge context window — MiniMax's is one million tokens — but the question is whether it helps you reach a correct result faster. Some passages in a report matter much more than others for a given question. Sparse attention first runs a small selection step that scores blocks of text; the main attention then works on the selected blocks, while nearby text stays in view. The selection changes with the query, because different questions need different parts of the context.

For the research job, the aim is to cut attention work across many documents. But inside one document a short footnote may change the entire conclusion: for a cash-flow question, that footnote can matter more than pages of industry outlook. Less attention work still has to find the important evidence.

So the next question is whether the model still finds the facts it needs. M3.1 Flash uses sparse attention in its first three layers, which makes retrieval an important check. A separate test placed facts at different positions in long inputs; the model passed all 39 requests, the longest about 830,000 tokens. That covers the test cases, but the research job needs checks on real reports: can the model find a cash-flow footnote, use the right reporting period, spot a conflict between documents? Move the important passage around the input and ask the same question again. A long input is useful only when the model can find the key fact.

Next come the numbers inside the model. The model keeps internal information from earlier text in what is called the KV cache. M3.1 Flash stores its keys and values in four bits instead of eight, while queries stay at eight bits. That reduces the data stored and moved, but it also changes the precision of that internal information.

In MiniMax's evaluation the macro average is almost unchanged, yet SciCode drops 6.6 points while other scores rise. An average can hide exactly the mistake your users care about. And comparing two complete setups does not say which change caused each score. For the research job, check the output again: are the sources right, are the calculations right, does the explanation match the evidence?

The same idea applies to the experts. M3.1 Flash is a mixture-of-experts model that selects some experts for each token, and those routed experts now use four-bit weights and activations; the shared expert is outside this change. Lower precision reduces data movement and allows more efficient computation, but the calculation has several steps — scale the numbers, compute, restore the scale — and where the scale is restored decides whether the answer is right. Fast math still has to be correct, so MiniMax measures the complete implementation, reruns the same research questions, and inspects why answers still miss the checks.

Having looked inside the model, Morgan turns to how it generates an answer. Speculative decoding works in two stages: a faster draft process proposes a short sequence of tokens, the target model checks it, the system keeps the part the target accepts, corrects where needed and continues. Sometimes several proposed tokens are kept; sometimes the draft does work that is thrown away. The opportunity is to get several tokens from one target check, but drafting, checking and rejecting all take time, so count the time spent checking the draft too.

Those checks are on tokens; financial claims and sources need their own. For the research job, include source retrieval, tool calls and retries. The measure that counts is the time until the analyst has a report they can really use.

Even when a single request runs well, many users change the experience. In a separate serving test with low reasoning effort, concurrent requests rose from 32 to 64. Total output per second grew about 1.9 times, but the P95 wait for the first token rose from about 1.5 seconds to over 11 seconds — roughly eight times longer — even though the median stayed acceptable. Users feel the wait, so a high throughput number is not enough. And the first token may still be reasoning: what matters is when the usable report arrives. Set a deadline for that report and measure how often you miss it.

One more setting changes the work the model is asked to do: reasoning control, used for the first time in this model. Thinking stays on, and if the effort is left blank it defaults to max, so the default alone can change how much work a request asks for. A coding test shows the trade-off: high effort used fewer output tokens than max, but accuracy fell from 80% to 68%. Fewer tokens came with lower accuracy, so give each task the thinking budget it needs.

For the research job, compare these settings on the same reports and check whether each one finds the evidence and supports the calculations; a quick first pass and a report ready for review may need different budgets. Record which reasoning setting you used so later work can rely on it.

To ship the research workflow with all these changes, first fix the question, the source reports and the information cutoff, and agree on what makes a report acceptable before running any comparison. Then check three things. First, can the analyst trust the report — open the sources behind important claims, check the reporting periods, recalculate the numbers and look for evidence that could change the conclusion?

Second, what did the finished job cost and how long did it take? Include retrieval, tool costs and retries; if a person must fix the report before it can be used, count that work too, along with every failed attempt. Test with the number of users you expect, check whether reports arrive within the promised time, and measure the cost of a report you can actually use.

Third, can the result be repeated? Save the model and runtime versions, the prompt settings and the sources, and keep examples of failures. Start with a small rollout and keep a way to switch back.

Morgan closes with a short recap video made with MiniMax's own video model, and mentions another, open-weight model the company is working on for text-to-video.

Back to the opening question: should you switch to a faster model? Look at the finished job. For the analyst, that means a report with sources they can open and numbers they can check, arriving when they need it. M3.1 Flash changes how computation is spent; the complete task shows what those changes mean for the user. Measure the cost of finishing the job on time. Faster models really matter, but what matters most is what we can do with them.