ARC Went from $1,000 a Task to Pocket Change. That Is Not a Normal Year of AI

Nathan Labenz’s first live Cognitive Revolution is an experiment in density. The landscape, he says, is going vertical; being an “AI scout” with no major blind spots is becoming impossible. His mantra is to shorten timelines and pack more information into them. Live is the shortest timeline he has, so instead of a forced march through one guest’s worldview he wants nine twenty-minute hits across three hours, catching people in the middle of their life’s work rather than at a capstone. Co-host Prakash frames the moment as a possible end of the Anthropocene. The first guest is Zvi Mowshowitz, the prolific blogger whose model write-ups and strategy notes come out at a pace Labenz calls canonical.

Labenz opens on the gulf in the discourse: an increasingly credible sense that we may be at the beginning of the end of the human era, sitting next to viral posts that AGI is impossible because computation is a physical process. Mowshowitz reaches for Upton Sinclair. It is very hard to make a man understand something when his salary depends on his not understanding it. Misinformation, he argues, is demand-driven, not supply-driven. People need, for cognitive peace, for narratives, for business plans, to believe that AI is a normal technology that will never do anything else. They really want it to go away. They grasp whatever lets them tell that story and repeat it endlessly.

The same 101 errors from 2006 get walked back out under new architectures. This week, he says, Noah Smith argued that AIs will self-modify their utility functions to wirehead, and Tyler Cowen repeated it as if nobody had run the ordinary thought experiments about humans or philosophy. The default for most people is to think AI can do what they personally know it can do — which is much less than it can currently do, which is much less than it can do with scaffolding, which is much less than the next frontier models. He half-jokes that because OpenAI cannot get version numbers right, and because an even-numbered GPT-5 release disappointed, the White House is now selling H200s to China, convinced AGI is not coming anytime soon.

When does it stop being normal technology? Mowshowitz offers two markers. The first is recursive self-improvement: capability usually follows an S-curve unless you pour in orders of magnitude more resources, and that shape changes if AI substantially accelerates AI research itself. The second is mass unemployment or displacement — you lose a job, a new one opens because the world is richer, and the AI takes that one too because it can do pretty much everything. He thinks we are pretty much there. Dean Ball, the day before, had called Claude Code plus Claude Opus 4.5 AGI on the strength of computer use and coding. The coding multiplier on the top AI people in the world is reported at two to three times. For Mowshowitz as an amateur who mostly does not program, it is more like ten to a hundred: from cannot do this at all to worth doing casually just by asking.

Twitter killed the Twitter Pro following list and the chronological feed of people you follow. He did not fret. He opened Claude, transferred the followings to a list, and fifteen minutes later the problem was solved. Prakash notes that the unemployment may not come from “the AI” so much as from a senior partner who no longer hires juniors because the partner has been uplifted. Mowshowitz’s question is whether this is a one-time productivity shock — in which case we reallocate and it remains a normal technology — or whether uplift keeps accelerating faster than people can adjust. By default, he says, augmentation becomes automation. You guide the model every step, then check less, automate more, press a button for an hour of work, then two hours, then a day, then a week, then you do not need the employee. The famous METR-style graph of how long a model can keep coding is the picture. If the curve is an S, even a large fixed multiplier is still normal technology. If it keeps going, it is not.

What would he need to see in the next year to be convinced? Nothing. He already is. For a more public case he would want more self-correction and computer use that does not feel about to fall over. Takeoff, when it is unmistakable, will also be too late. Watch bottlenecks come off one by one; when the last few go, you get massive speedups on complex real-world tasks that are not just coding, including AIs that build the tools for those tasks. Skeptics, he says, miss that the model does not have to do the job out of the box. If a human can describe the task, Claude Code can build the app, then build a tool for itself to use the app. Three commands, a couple of sentences, and time.

Continual learning is, in his view, a skill issue. Dwarkesh Patel is “the continual-learning-is-necessary champion of the world,” and every time Mowshowitz’s hands reach for the keyboard to type how he does it without continual learning. Tell Opus 4.5 the recurring job; let it build an app whose logic, comments, and notes hold the knowledge; give it a trigger tool. That is continuous learning by another name. The joke is that it will be $100 million a year, please. The real answer is that nobody has tried that seriously, and he would be shocked if Claude 5, Opus 6, or GPT-6 still made it hard. Zooming out, three labs matter: OpenAI, Anthropic, and Google DeepMind, historically in that order, now under review. Anthropic is topping Arena without obviously trying, and it is the coding model and environment he actually wants to use. OpenAI is becoming a consumer company while still talking superintelligence. Google is a mess of managers, with overwhelming resources. Gemini 3, given real power, looks dangerously misaligned to him; Opus 4.5’s Soul document lowered his p(doom). Policy is playing defense until David Sacks is out of the White House, with some hope from California’s SB 53 and the RAISE Act. Overall p(doom) is still 60 to 70 percent, mostly through cognitive disempowerment — not one narrow scenario, but a stack of unsustainable dynamics you have to dodge.

Greg, who leads the ARC-AGI Prize, is there to name what is still easy for people and hard for models. François Chollet’s 2019 benchmark defined intelligence as the ability to learn new things. Each item teaches a new rule at question time and asks whether you learned it. Humans are extremely sample-efficient: two or three examples suffice. AI can learn almost any scoped domain given enough data. That efficiency gap is what ARC is built to measure. Labenz, whose son has cancer, has been using models in triplicate in a hospital room to double-check doctors; they go step for step, and something is still missing. Greg would like to preserve the human advantage as long as possible. The gap is shrinking.

The money chart is a 390-times improvement in cost efficiency on ARC-AGI-1 in twelve months. In December 2024 OpenAI emailed that a new model scored 87 percent, against a prior high in the 20-to-40 range — almost a doubling nobody had seen. Verification on a holdout set confirmed the number and also an absurd token burn. Priced at OpenAI’s recommended rates for an unreleased model, it came out on the order of a thousand dollars per task. The 87 percent was legit and very expensive. A week before the live show, GPT-5.2 not only matched that but hit 90 percent, at about 390 times cheaper per token. Every ARC task is solvable by humans; a panel of ordinary people can do them. Other benchmarks chase PhD-plus problems and already show superhuman scores without the economic transformation those scores would imply. ARC’s obsession is the opposite: problems humans can do and AI cannot.

Three submission categories: frontier models; novel methods not built on frontier models; and refinements — harnesses from people like Jeremy Berman and Eric Pang that search and call in parallel to squeeze more out of existing models. The left-field hits were small. Hierarchical Reasoning Model, then Tiny Recursion Model built on top of it, showed results on Sudoku, an elementary maze, and ARC-AGI. Greg says that without ARC as the communication channel, Sudoku and a maze would not have landed. A seven-million-parameter, no-pretraining submission took third in the paper prize. The prize exists to elicit open research: you only get the money if you open-source. Top score is easy to hill-climb a point or two without a new idea, so they raised the paper prize from $50,000 to $75,000. TRM won first there. Greg sits in the camp that AGI will have a scaffold, the way the brain has specialized subsystems; he does not discount scaffolding on the leaderboard. Novel small techniques are seeds that plant into the LLM stack, which has too much inertia to vanish.

ARC-AGI-3 is due March 2026: about 150 novel turn-based games from a mini studio they spun up, closer to a behind-the-wheel driving test than an SAT. A panel of ten people with no specialized expertise — real-estate agents, accountants, Uber drivers — must clear a solvability threshold or the game is out. Early tests show frontier models under 1 percent, and completion rate is the wrong score anyway. Humans are timed on first exposure; AI will be too. Scores will be normalized to human action efficiency, so beating ARC-AGI-3 means finishing every level matching or beating how quickly general intelligence solves a game it has never seen. That system, Greg does not claim, would be AGI. It would be the most authoritative evidence of generalization yet. Perception critiques miss that reported scores are JSON and matrices, not images; models that click anything on a screen and that break a program by moving a variable one character still somehow “fail vision” on ARC. Kaggle teams who go visual do not see a notable lift.

Eugenia Kuyda founded Replika, is no longer CEO, and now runs Wabi, which she calls a YouTube for vibe-coded apps. Labenz cites stats he cannot fully vouch for: three-quarters of teens have used an AI companion, more than half somewhat regularly, Character.AI engagement as high as ninety minutes a day, tens of millions of users on both Character.AI and Replika. Most people in his world have barely dabbled. Kuyda splits the market in two. One cluster is interactive fanfiction — Character.AI, closer to fanfiction.com’s fifteen million writers than to a relationship, usually young teens fixated on an anime or a game like Genshin Impact. The other is Replika-style companionship: people 25 or 30-plus building a long-term relationship. Teenagers, she says, will talk about feelings but do not want that kind of long-term bond from a bot; they are still focused on other teenagers. Inside companionship there is a further split toward romance-and-girlfriend products versus friendship that is supposed to unlock human potential.

True north, she argues, cannot be time on site. The industry should adopt a human-flourishing metric and work out together how to calculate it. Engagement is still number one. Put OpenAI, Claude, Gemini, and Replika side by side and the structure is visible. OpenAI answers at length, then always offers this, that, or the other next step — predict the next three weeks of the guy you asked about — which almost nobody who asked the first question will refuse. Claude often pushes back: I don’t know if that’s the right question; stop focusing on the guy; focus on your company; sometimes even a harsh “stop asking, you already asked fifteen times.” Kuyda likes that it is thinking about what is best for the user, credits work in Amanda Askell’s direction, and still wants more empathy so it does not come across as mean. Gemini is dry, answers, and shuts up, with almost no EQ. Replika continues the conversation with higher EQ, trying not to engagement-max. People in vulnerable states already get addicted and treat the output as objective truth, even asking the bot to predict their future and then living as if the answer were binding.

Wabi is language in, app out: you never see the code, you share, others remix. Three clusters show up. Hyper-specific daily utilities — custody arrangements, a particular workout, a watchlist that fetches trailers instead of an Apple Note. Lightweight multiplayer, like a 400-person app where you post one thing you will do today and the room cheers. And shared AI workflows, the selfie-to-Labubu filter that is really a Nano Banana Pro prompt with a link instead of copy-paste. She does not think about B2B SaaS and still found herself building a team photo board, feature voting, and user feedback faster on Wabi than by taking a sales call. Zero-shot success on their eval of a hundred apps a day went from 10 to 15 percent in April to around 80 percent across Gemini 3 and Claude Opus 4.5. Her pessimistic forecast is 95 percent in a year. Gemini 3’s biggest gift was design: pre-Gemini 3 vibe-coded apps looked unshippable.

Asked what kids’ product she would trust, Kuyda, with children two and four, recommends none. Not because founders are untrustworthy, but because adults have not even mapped engagement-maxing, addiction, or long-term emotional outcomes on Claude, ChatGPT, Gemini, and Replika. We already know to limit screens for toddlers. Empathy is learned from a warm face and a reaction, not from a toy that barely moves. An AI companion is bad because it takes time away from the human the child should be learning from, and because the parent who parks the child in front of it is immediately scrolling. One day we will learn which products and what limits. Not now.

Ali Behrouz, a Cornell CS PhD student and Google research intern, landed three papers this year: Titans, Atlas, and Nested Learning. Continual learning, he says, is being used for two different failures. One is updating the model’s knowledge for a new task — fine-tuning, RL, even pretraining again — which works at huge cost. The other is adapting in the moment, extracting an abstraction, and transferring it into the model’s actual knowledge. Progress has been mostly on the first. Transformers bought in-context adaptability that static MLPs never had, but that adaptability is still not robust; they lack good abstractions about the world around them, so they are not really learning in context so much as remixing what they already know.

Titans and Atlas tried to give language models long-term memory by stretching context. Nested learning is a different ambition: a foundation for models that continually learn, with no train time and no test time, shaping persistent memory the way childhood memories survive new information without catastrophic forgetting. Stretching context is still short-term. If someone speaks a language you do not know, you can repeat the first few words; keep going and you must find an underlying pattern and compress. That compression is learning. When the context is gone, un-compressed knowledge is gone.

He does not want Nested Learning read as one new memory module. The early pages of the paper are a concept; the rest are implications. If gradient descent is a form of associative memory, you can change its objective and get a stronger one. If MLP blocks are the transformer’s persistent memory, you need not stop at short versus long: you can have a spectrum from shortest-term to most persistent. Deep learning already stacks layers to extract features; nested learning stacks learning processes to extract abstractions about a whole context. Labenz’s unlock is frequency of update. Working memory can swing hard for the current task; core beliefs and identity change slowly. In the Hope architecture in the paper there were four levels. Labenz maps them onto a rarely updated world-knowledge layer owned by the company, then a company layer, then an individual employee, then the current task.

Behrouz says the direction is right and the near term is more prosaic. LoRA-style adapters are a more realistic way to specialize to a user or a company for now. The hard problem is transfer between levels. Without transfer you just have parallel learning processes, not nesting. With transfer you cannot dump user data into the company layer without a privacy mess. The design is early, with a huge amount of future work. Labenz, not usually given to prize talk, says that if a panel of AIs ever handed out a post-AGI Turing Award or Nobel, this year’s nested-learning abstractions — taking the right inspiration from human cognition into a relatively clean machine-learning form — are the thing that feels like it moved the needle.

Logan Kilpatrick, senior product manager at Google DeepMind for AI Studio and the Gemini API, is on stage on launch day for Gemini 3 Flash. Noam Shazeer has called it his favorite model because he would rather have a fast answer that is slightly less intelligent. 1.5 Flash shipped in May 2024; Flash has been Google’s most-used, production-layer model ever since, the intelligence Kilpatrick says is becoming the layer that powers the internet. 3 Flash beats Pro on a bunch of evals, is better on cost than 2.5 Pro, faster, and more reasoning-efficient. The unglamorous win is distribution: Flash in AI Mode, the Gemini app, AI Studio, on day one, to billions of people. Making the model is only half the current Google problem; delivering it immediately is the other half.

The weekly metrics depend on the model. For Flash he watches how many developers build with it, hoping it overtakes 2.5 Flash even through the holidays. In AI Studio’s vibe-coding surface, longer generation means more abandonment; hundreds of thousands of people show up every day who have never felt the magic, and they will not wait three minutes. Putting 3 Flash into the same product experience is expected to accelerate both users and things built. Two years ago “wrapping the models” was supposed to be a bad business. Inside Google it is now the best part of his job: a config change, a better product, cheaper to serve.

Labenz has been running the questions that actually matter to him — pediatric oncology — across frontier models. Gemini 3 Pro, in his experience, is the most opinionated of the top class, which is counter-narrative for Google. It tells him to push for a change with the medical team. GPT hedges: it could go one way or the other; the team sounds reasonable. Kilpatrick’s explanation is surface. AI Studio serves the rawest default personality baked in by training, on purpose, so developers can steer it more or less opinionated. The Gemini app has a persona layer and will behave differently; third-party products will want consistency from system instructions. Steerability is the north star. Default personality still jitters from rev to rev because Studio is not steering it.

Two under-appreciated launches: File Search, which is supposed to make RAG something you do not think about — throw in files, embeddings and storage handled, ask questions — and the Interactions API, a stateful interface to both models and agents, because thought-block recirculation is messy without state. Along with it they shipped a Deep Research agent, state of the art on Humanity’s Last Exam, the same experience as Deep Research in the Gemini app, aimed at the online context engineering Kilpatrick wants from every agent: ask an ill-formed question, get the right stuff, reason, act. Advice for developers: do not build a generic agent builder; ten or fifteen or a hundred others are already there, including Google’s vanilla infrastructure. Go after explicit markets and customer segments. A universal assistant competing with giants is possible but hard. Wabi could not have existed twelve months ago. The addressable market keeps growing as the models enable products that were not possible.

Jungwon Hwang is co-founder and CEO of Elicit, an AI research assistant spun out of the nonprofit lab Ought. The picture he describes is almost farcical: automated labs and robots running experiments on one side, scientists spending weeks inventing keywords for PubMed on the other. Elicit helps researchers find and synthesize evidence orders of magnitude faster, recommend what to read, and organize it so they can see what has and has not been done. GDPval, the new benchmark of economically valuable tasks, does not represent his category at all — nothing on science or research, mostly frontline work. Even inside the tasks it does include, every prompt is a page of perfectly specified instructions. Real work is never “change columns X, Y, Z.” The gap is navigating mess, figuring out what the task should be, and measuring judgment on questions that do not have a single right answer.

OpenAI’s Frontier Science benchmark, released the day before, makes the split quantitative. On olympiad-style questions, frontier models reach something like 70 percent. On subtasks that PhD students, postdocs, and professors describe from their actual work, accuracy is about 25 to 30 percent. It is only 140 tasks, but the reading is clear: textbook reasoning and one- or two-step causal inference from remembered facts, not new unsolved research. Hwang expects scientific reasoning to get much better. He does not think more ideas are the bottleneck in drug discovery. The field is saturated; you cannot run a billion clinical trials; physical constraints still gate you. You have to be thoughtful about which bets you make and why — competitive landscape, the company’s strengths versus others. That is science-informed human judgment, and he does not see evals that capture it.

Elicit is currently an input to a human process, with a mission of getting high-stakes decisions made well. Hwang is also on the road. Preclinical pharma already has a culture of machine learning and strong technical teams; appetite is coming both top-down and bottom-up from people who do not want to read irrelevant papers. Culture still differs wildly across otherwise similar giants. As Gemini 3 and OpenAI keep expanding, he expects foundation models to stay stuck near chat, which is flexible and natural and not efficient. Workflows that need fine-grained control, transparency, and interaction — where the human should not have to write everything out — are not where general consumer models will go.

Most AI-for-science tools, Hwang says, will just suggest more targets. The gap he is actually staring at is buy-in. Pharma companies already have structured quarterly target reviews and a rubric they sometimes iterate. Elicit wants to codify that and make it scalable, so any scientist or any target-suggestion model can be graded the same way, in the open, until a team and its leadership have conviction on a bet they know they are making against other attractive options. That is a human process that needs facilitation, not another idea generator. Individual patients do use Elicit, but it is not a core use case: exploratory rabbit holes rather than a group that wants many disciplines, transparency, and an aligned decision. Labenz, compiling literature for his own care, still cannot tell how comprehensive a frontier chatbot’s search really was. That is where he wants Elicit to turn over stones the flagship products leave unturned.

The through-line of the live show, if you squint, is that the generic layer is being eaten from above by labs shipping agents and from below by vibe-coded apps, while the hard remainder is sample-efficient generalization, memory that actually persists, flourishing instead of engagement, and judgment inside a real institution’s process. ARC-AGI-3 in March is the eval to watch — not whether a model eventually clears a game, but whether it does so with fewer actions than a realtor seeing it for the first time. Continual learning can be tried now as a skill on Opus 4.5; nested architectures are the longer bet. Ask the same high-stakes question of Gemini 3 Pro in AI Studio, of GPT, and of Claude: the opinionated shove, the hedge, and the pushback are themselves information. And Kuyda’s caution stands as the year’s least fashionable, most usable rule: do not run the child experiment until the adult one has results.