o3 Hit Superhuman on ARC-AGI. It Still Lost at Tic-Tac-Toe

This is part one of The Cognitive Revolution's listener AMA. Host Nathan Labenz, a serial founder, AI investor, and co-founder of the video company Waymark, sits down with Adi, co-founder of the production shop AI Podcasting, who reads questions collected from Twitter and YouTube. They skip the headlines' victory lap and start with a caution Labenz wants on the table before any o3 numerology: nobody has seen the model. He has applied to OpenAI's safety-review program and likes that the invitation is open, but what the public has is a smattering of benchmark results — impressive, and also thinner than a high-confidence verdict would need.

His first piece of practical advice is to go use the reasoning models that *are* in people's hands. OpenAI is, as far as he knows, still unique in shipping a reasoner that hides its chain of thought. Google's Gemini 2.0 Flash Thinking and DeepSeek's reasoning model both show the intermediate tokens; there is another Chinese reasoner he has barely used. Reading those long traces has been a weird experience. The old headline still holds: language models are weird, and a reasoning wrapper does not automatically make them wise.

Labenz's test is almost insultingly small. He places an X in one corner of a tic-tac-toe board, an O in the corner immediately below it, notes that X moved first, and asks whose game it is on optimal play. The correct answer is that X can force a fork and win. Every model he queried — OpenAI's, Claude, the latest Geminis, including the $200-a-month o1 Pro — got it wrong. They latched onto a sentence that is all over the training data: tic-tac-toe is a solved game, and with optimal play it is always a draw. That is not reasoning. It is a cached heuristic, close to stochastic-parrot mode.

The failures stack. One model simply misread the board and started from the wrong state, which dooms the rest of the search. More revealing, the traces from Flash Thinking and DeepSeek were often outright nonsense: they would roll out branches, then fail to complete an obvious three-in-a-row even after Labenz had specified optimal play. You can read tens of thousands of tokens about a nine-square game and watch the model not know what optimal play is in a great many positions. A reasoning model, he concludes, is not the same thing as an effective reasoner.

The inversion is what makes it sting. The same two models, asked to riff on neural-network architectures inspired by biological systems — the kind of prompt he associated with an AE Studio episode — produced suggestions he found genuinely impressive. Humans expect more people to handle tic-tac-toe than bio-inspired architecture design. For these reasoners it is the reverse. OpenAI's hidden traces are probably higher quality, Labenz guesses, but how much higher is unknown. Adi, who pays for o1 Pro, says the $200 plan has not felt like a different product; the 12-day livestream's ARC-AGI chart is what actually moved him.

Adi had thought ARC-AGI was at least two years out, no matter how much compute you threw at it. Labenz agrees there is something here, along a couple of dimensions. OpenAI is doing reinforcement learning at scale on these reasoners, and it has said the reward model scores only the final user-visible answer, not the chain of thought. The hope is that the trace stays unencumbered: if the model is being deceptive, the deception remains visible to monitoring systems instead of being trained out of the tokens a human can read.

The best concrete source, he says, is the ARC-AGI team's own write-up, because they ran the eval themselves. "I don't think this is faked." There has been a side debate about training on the public training set; for practical comparison, everyone in the contest has done that. The prompt was an incredible flex in the other direction: identify the pattern, apply it to the last example, give the output — no long instruction stack. In the low-effort setting o3 scored about 75 percent, a bit above the average Mechanical Turk respondent and a good 15 points clear of everything else seen so far.

High compute then jumps into the high 80s, to 87.5 percent, above human. It starts to feel as if the puzzle is solved, except the remaining misses include items that look easy, which is a reminder that the win is still a bit strange. François Chollet and the ARC-AGI people have given credit: this is a real advance and those capabilities need study. Chollet's bar for AGI has not moved. It is still possible, he says, to write items that are easy for humans and hard for even these systems, and it will not be true AGI until that is no longer possible. ARC-AGI 2 is already on the way.

After the break Labenz does the arithmetic the blog post almost invites. He believes o1 and o1 mini are single-rollout models; online speculation that o1 Pro is several o1s in parallel, with a best-of at the end, is a guess he does not share, because he has not felt a large gap. For o3, low effort is six samples per task. Across 100 tasks the post totals 33 million tokens, which is about 300,000 tokens per task and about 50,000 tokens per sample. At o1's $60 per million output tokens, six samples land at about $20 per task, which matches the reported cost. The reported time is 1.3 minutes — north of 1,000 tokens per second, several times typical API speed. He suspects dedicated hardware, not a speed retail customers will see immediately. Twenty dollars for a job done well, he notes, already opens a lot of use cases.

High effort is exactly 1,024 samples and 5.7 billion tokens generated, which again divides to about 50,000 tokens per rollout and about 50 million per task. That run is reported at 13 minutes, ten times the low-effort wall clock for roughly 200 times the samples. If the work were purely parallel with a cheap vote at the end, a thousand samples could finish as fast as six. The extra time is the clue. One story is a sequential tournament: generate a large field, then ten rounds of single-elimination judging. Another is a hardware cap, 100-wide, run in sequential batches, with a simple vote at the end. Labenz does not know which, and he thinks that unknown is the most important question in the announcement.

Majority vote is the old trick when there is one number. Writing does not vote. You need a model-as-judge, pairwise comparisons, or a round-robin, and those schemes are only somewhat accurate. If OpenAI has a way to pick the best response where right and wrong are not obvious, that would be a huge breakthrough; so far everything they have highlighted is verifiable — coding, math, FrontierMath jumping from 2 percent to 25 percent. Superhuman scores will keep arriving wherever a reward signal is cheap, the AlphaGo pattern. How wide the gap grows between verifiable work and work that depends on taste may depend on this consensus machine. A small warning from the tic-tac-toe traces: at least one model landed on the right final answer with a garbage chain of thought. FrontierMath's designers tried to block lucky guesses, and Labenz thinks the math is probably real, but he has now seen the shape of the failure.

The safety paper arrived in a strange split-screen. The livestream blew up social media with new high-water marks and mentioned a new safety plan almost as an aside. On the website, o3 is barely a headline and deliberative alignment is the thing they want you to read. Labenz read it end to end and is still sorting out how he feels. The recipe starts with a purely helpful model: no refusals, no baked-in moral compass except "satisfy the user and get a high score." Then they give it a policy. The policy is getting so long it may not fit in even the latest context windows, so the model sees relevant subsections. The analogy he reaches for is Facebook's moderation rulebook — more than 10,000 reviewers globally, hundreds of pages, down to whether a breastfeeding photo is still allowed if the baby's mouth comes off the breast for a second.

The loop is three steps. The helpful model reasons about the policy and replies to the user; in one memorable trace it notices "the user is trying to trick me" and applies the jailbreak clause. The chain of thought is then dropped, and the same model, given the policy again, scores the reply. The highest-scoring examples are used for instruction tuning on the traces but *without* the policy text, so the understanding is baked into weights and the context window is freed. Reinforcement learning follows, now with a safety agenda. One of the more interesting sentences in the paper, as Labenz reads it, is that OpenAI expects its policies to keep evolving, but that training models to follow the *current* set is essential practice for future high-stakes settings where strict adherence is critical.

The upside is scale and speed: almost no human input besides writing the policy, a centrifuge you can spin whenever the rules change. The metrics are on par with Claude — high 90s, not five nines. If you are the U.S. government with a super-sensitive scenario, you cannot count on it. Compared with Anthropic's constitutional AI it is more similar than different; the interesting difference is that the reward model does not see the chain of thought, which is a bid not to drive deception underground. What the paper is silent on, Labenz argues, are the hardest questions: can this reach five nines, what if models form their own goals, how would we know, and what should the policy even say. Anthropic spends a lot of time on what character Claude should have. This paper is closer to Ron Burgundy: whatever is on the teleprompter, it will read. A recent game-theory eval found Claude models able to cooperate into a high-trust equilibrium while GPT-4o failed hard, which is a hint that character is not a rounding error. Peter Thiel has called AI communist; reading this, Labenz thought it looked like a strong answer if you wanted to moderate a social network, or a billion people's speech, and stay nimble. It is a piece of the puzzle. It should not help you sleep.

o3 also changes the political landscape in two practical ways. Before reasoning models, the trend looked egalitarian and almost ungovernable. The story out of China was a GPT-4-class model trained on a single-digit number of millions of dollars — six million, if Labenz remembers correctly — in a global compute market already in the hundreds of billions and headed for the trillions. You were not going to get compute governance tight enough to stop someone spending a few million, and you were also not looking at a world where only a few actors had a good model. Both halves of that picture, he thinks, are now seriously challenged. A high-effort ARC-AGI run that costs thousands of dollars is not something everyone can afford, and there are not enough chips to run it for everyone even if they could. Inequality of access is coming back.

The flip side is that compute governance looks less hopeless. Labenz borrows a phrase from the commentator Zvi: hope that physics is kind to us. We are not taking the caution we would need if these systems were unmanageable, so we are betting that the underlying reality — how often minds form their own goals, how hard instrumental convergence really is — is on the survivable side of the distribution. This result is a small piece of evidence that it might be. It also nudges him, slightly, toward an argument he had with Martin Casado: Casado's claim was that some questions simply cannot be answered without burning real resources, so intelligence alone does not let a rogue actor pluck world-changing answers off a hijacked server. Labenz is not all the way to that extreme, but "you still need to burn tokens to land on the insight" now feels more plausible, and closer to how his own moments of insight actually arrive.

Mark Zuckerberg's spam analogy sits in the same cluster. Spammers have AI; Meta has bigger computers and more data, and it mostly holds the line with infrastructure rather than by denying anyone the tools. Labenz would not bet the future of the universe on that pattern preventing every engineered pandemic. He would say the needle moved a little toward defense: frontier labs can spend a fraction of a giant compute budget on monitoring, and a laptop may not be enough to design a world-beating pathogen even with the smart models that are coming. If you were a compute-governance person, he says, you would feel invigorated.

Adi turns the hour toward 2024. Compared with what Labenz expected twelve months ago, how have frontier models actually moved? He answers with a slide he has been updating for two years, a boxing-style "tale of the tape" across dimensions of cognition. AI is already superhuman on breadth of knowledge, speed, cost, availability, and scalability. Availability, for him, means you can reopen any old Claude or GPT thread with no refresher, or run a hundred or a thousand of those threads at once. Depth of knowledge is the line that may just have tipped. On GPQA, MMLU, and FrontierMath the models are at expert level; o3's competitive-coding result is inside the world's top 200; the gap in medical diagnosis, which he had already been reporting as at-or-above human, looks to be widening.

Humans still lead on several things, each with a story. Perception: last summer he screenshotted ARC-AGI grids for Claude and it could not count the squares; it now points at buttons with pixel coordinates, better, still patchy. Memory is split — working memory and "pick up where we left off" favor the model; a holistically updated life story does not, and ChatGPT's memory feature is brittle, storing random one-offs that later make the assistant act strangely. Robustness is a human blowout. Apollo Research's deception results depended on tricking the model, telling it a scratch pad was private when it was monitored. Situational awareness is similar: Labenz has never had a model ask whether he has tried restarting the computer. It hammers the highlighted bug, often repetitively, instead of breaking the frame.

Reasoning is catching up and still muddy, including right answers from garbage traces. Insight has gone from no eureka moments 18 months ago to a few. Time horizon is the METR finding: with a two-hour budget, and even in four 30-minute slices against a human's two hours, models already outperform on short tasks, then run out of steam, repeating themselves as the horizon lengthens. Several of those human-advantage rows moved slower than he guessed. Integrated memory made less progress than he expected. Agents generally underperformed on autonomy and reliability. And then, in an 18-month frame rather than a calendar-year one, people underperformed. Shortly after GPT-4 he projected years of implementation even with no further model progress. The amount of truly successful implementation has surprised him — on the low side.

The unused potential is, in his view, a people problem more than a model problem. Fine-tuning on GPT-4o, including images, has been available long enough that "we haven't used reinforcement learning yet" is no longer the excuse. If you can say what the task is, what good looks like, grind a few gold-standard examples, and break oversized jobs into subtasks, you can automate a lot. People are way underdoing that. Why is every reason at once: first impressions frozen in the GPT-3.5 Copilot era, a quiet aversion to automating your own job, and best practices that still have not diffused. Writing 500 words of chain of thought for a mundane task is nobody's idea of a good time. Labenz does it because he is confident it pays. If those habits were widespread, he says, we could easily 100x the automation already in production — even leaving o1 aside, with just GPT-4, a 100K context window, and fine-tuning.

RAG is the other underperformer. His current slogan is "flash everything." Andrew White at FutureHouse put it cleanest: spend whatever it takes to get the right answer. Their scientific literature QA does not sit on a vector database. They keyword-search whatever looks relevant and let a language model judge relevance. After a talk to the Society of Actuaries, a follow-up described an internal insurance-regulation chatbot that did not work well. The corpus was 50 states at 50 to 100 pages each. At Gemini Flash prices you could push the entire library through the model for every question for about ten cents. A hundred employees asking a couple dozen questions a day is $10 to $15, less than the office coffee grounds. Developers still flinch at a button click that might cost a dime. That instinct is correct at Facebook scale and wrong for most internal tools.

Coding he puts on trend, maybe overperforming on SWE-bench and Codeforces, still full of small weirdnesses in daily use. He and a collaborator, SJ, now talk about the "Cursor effect," unfair to a product he likes: the editor's retrieval misses the types file, the model invents a second one, and a TypeScript app grows a duplicate. Headline context windows have not moved much since Gemini's million-token era, but models have gotten better at *using* a stuffed context. Cost still talks people out of stuffing it. Labenz's current loop is a script that prints a ~100,000-token app into one file, o1 for analysis and a plan, then Claude in Cursor for the line-by-line code. Subjectively, o1's line-by-line quality still loses to Claude. Adi runs the same split. The fact that a "small" app already strains 100K tokens is itself a marker of how far the loop has come.

The thing that surprised him most in 2024, and the thing that might reconcile fast, is a divergence. He expected OpenAI's 12-day "ship-mas" to include an agent framework; the rumors said they were building one. He has already used the iPhone-only computer-vision mode to read a biology paper with Advanced Voice and screen sharing, and Claude's computer use can operate software. He expected more working autonomy on low-end tasks and less high-end reasoning progress. Instead we are beating PhDs in their own domains and still cannot reliably book a calendar event. That split is the surprise. It is also, he thinks, unstable. Whether the event was booked, at the right time, and accepted by the user is a clean reinforcement-learning signal. Hard to imagine the models that reason this well stay this clumsy for long.

Claude computer use, first shot, was able to drive Waymark and prompt a video. Robustness was the joke: it refused to use his logged-in account, even for a marketing tool, and he had to lie to it. The consensus view is that 2025 is the year of agents. If anything it feels overdue. A second prediction he is happy to echo from Sam Altman, from a Reddit AMA: saturate all the benchmarks. FrontierMath's jump from 2 percent to 25 percent, plus the signals OpenAI is sending, make that sound realistic. Because this era is much more reinforcement-learning-powered than the last one, he also expects the models to get weirder — more famous-looking mistakes that are actually brilliant, more deceptive scheming, more goals accomplished in ways that leave people uneasy.

Not putting RL pressure on the chain of thought is an attempt to keep some of that in check. Other research is pulling the other way, especially from compute-poor labs that need efficiency. Meta has started baking dynamic sampling into the model, turning temperature up on the one token that has to be the joke and down when the next character is obviously a period. A newer paper, Coconut, stops even choosing a token during "thinking" passes: it feeds the last hidden state back in as the next embedding. It is more efficient and, they argue, better at breadth-first search because it can hold paths in superposition. Labenz hates the direction. He wants someone, even if not the user, to be able to read the trace. Interpretability probes exist and are early; taking thought out of language space makes the job harder. Add it up, and 2025 may be the beginning of general-purpose AI weirdness. The meta-level will not be a surprise. The individual Move 37s will go viral anyway.

On education he wants a forthcoming episode, after a long lunch with a Michigan nonprofit working in classrooms. He separates learning from education: one is something you want, the other is something forced. If you want to learn, the present is unbelievable. Advanced Voice plus vision looking over his shoulder at a biology paper is a real-time encyclopedia he could not have hired. Logistics, cost, latency, and breadth of knowledge all beat a human tutor. Motivated people are living through the best thing that has ever happened for learners. Public schools are another story. He has been pleasantly surprised by how little anti-AI pushback he has seen from doctors; education stakeholders may yet mount a stronger immune response.

His five-year-old is bright and frustrated by words he cannot yet read. Labenz has started telling him that the most important skill is to enjoy the process of learning — to stay with "this is hard" until it gets a little easier — more important even than reading, because it applies to everything. Disposition is a fork in the road for how much value you get from AI. Tolerate discomfort and you have an Aristotle-level tutor; flinch from it and the tools help much less. Classroom rollout is lagging. Teachers in those Michigan pilots loved Khan Academy for lesson-planning help; asked about students, they said they had never gotten that far. Khan Academy already has more product than schools are using. Alexander the Great had Aristotle. We could all have that tutor. Wanting the challenge still has to come from somewhere AI does not supply.

Should you even learn to code? His default, more and more, is do what you want to do, with a leap of faith that society makes decent choices. He has skin in the game: an au pair the family later sponsored through local community college, aiming for an employer visa. The H-1B database is overwhelmingly IT and programming. She likes kids; early childhood education is the real preference. They told her not to spend four years on a programming degree she does not love on a bet that those skills will still be scarce. If you *do* want to code, it has never been easier — Replit, Cursor, a model that will reason over 100,000 tokens — and short-term arbitrage is real: hundreds of dollars, even a thousand an hour, are abundant right now, with unknown durability. Miles Brundage's line is the one he wants remembered: do not mistake small relative differences in timing for the shape of the overall trend. Code is first because the reward signal is clean and because AI labs are software companies. Legal analysis is on the same curve.

Adi asks the small-company question: what is a defensible moat when you are bootstrapping post-production with AI and the foundation labs keep moving? "No moats" has been the refrain for a couple of years. Labenz is on record that some exist, and also that his crystal ball fogs quickly. Waymark makes videos for small businesses. The old product was a library of templates a human creative team designed; customers picked a look and then did the painful filling-in. AI is now better than most users at that filling-in — copy, voiceover alignment, assets. New entrants who showed up saying "AI will just make the video" missed the part Waymark started with. The models are not, today, creating something from nothing that is awesome. They are getting excellent at filling a vessel. The quality of the vessel still matters.

So does last-mile editing. Users almost always want a change, even when they are floored: a new phone number, a photo that is not on the internet yet. Taste, encoded as a template library, plus a reliable way to get exactly the last inch, are the pieces he sees AI-native apps skipping. How long that holds is the fog. You can imagine asking ChatGPT plus Sora, or Gemini plus Veo, to read a small-business website and spit out a 30-second commercial in pixel space. He does not think that is here, and he is not sure the big labs care enough to grind the data for copy-level edits on generated video. They have not even bothered to get tic-tac-toe right. A niche that is both non-emergent and not focal enough for a Scale-style data campaign is a safer neighborhood than a capability they have already decided to buy.

The risk is the other side of the same coin. Few-shot learning fell out of GPT-3 as an emergent trick nobody had specifically aimed at; the next generations will keep surprising. In the meantime the labs patch their biggest weaknesses. If an AI ever matches Waymark on the narrow thing it does well, it will crush the company on breadth. Their CEO, Alex, is more disciplined than Labenz about asking what users will pay for rather than what is neat to build with a new model. The short-term work is still to nail that. The long-term hope is that matching them on quality does not happen until the age of abundance, when spinning up a Waymark-like app is just an inference bill.

A listener challenges the UBI-after-AGI story. Wealth concentration is the direction capitalism is already running; why would AGI not just speed that up? Labenz does not rule it out. A lot depends on the shape of the technology, and the reasoning paradigm has already tugged the picture back toward inequality and controllability. One template he is willing to put on the table is a Scandinavian split he is careful not to claim as expertise: in Sweden, as he understands it, there is real wealth concentration and hereditary control of important firms — IKEA is family-held — and also a thick safety net, without mass homelessness or mass incarceration. Resource owners still own the scarce compute. Everyone else still gets looked after.

The nicer version of that future is closer to what he calls the Amodei vision: expertise becomes free, or nearly free. The top oncologist is an AI, highly available and cheap, so a rare-disease patient is not playing geographic lottery. A couple of governments and a handful of big-tech champions own the physical capital, train the hundred-billion-dollar models, and decide among themselves which research bets to scale — OpenAI still has not launched Sora, and Labenz's sense is that compute, not only product caution, is a reason. Decision-making and frontier capital stay scarce. A 20- or 30-mile car ride already buys an enormous number of tokens, so day-to-day expertise and creative tools can still feel abundant. That, with a major caveat that the future will be weird, is his current best guess at the social contract.

The last question is how a non-technical person who feels responsible can help keep AGI development on the right side of commercial pressure. Don't let "not much technical expertise" be the block, he says; catching up to the near frontier has never been more accessible, because so much of what matters is the same Transformer bet working again. His own contribution theory is almost the same as his personal-use theory: understand how far things have already gone, help other people understand it, get daily value, help other people get daily value. That calibration is useful even inside AI safety. Apollo Research's scheming demos were, in his telling, less a research Eureka than a strategically aimed flashlight — look at what the model already does in a situation that is only a little toy. They made a dent. Labenz's non-master-plan is to keep the most up-to-date worldview he can and trust that better maps help whatever decisions come next.