A $500 Kernel, a Ten-Month Village, and the Temper of Agents

Nathan Labenz and co-host Pash, known as 8teAPi, open part one of a four-hour Cognitive Revolution live show by bringing Stanford professor James Zou on stage. They had recorded a full episode with him not long before, and Labenz is still struck by the range: low-level interpretability on protein language models at one end, a high-level virtual lab of AI agents at the other. That lab had designed candidate nanobodies against new COVID strains. The check-in is whether any of it survived contact with reality.

Two more guests are waiting. Sam Hammond, chief economist at the Foundation for American Innovation, will talk geopolitics, Gulf energy deals, and why he thinks current AIs are at least as likely as not to be conscious. Shoshannah Tekofsky, on the technical staff at Sage, the nonprofit behind AI Digest and the AI Village, will report on months of watching frontier models live on full desktops with email, a shared chat, and almost no adult supervision. The topics look far apart. Underneath them is the same shift: models have started doing things, and the question is what to watch.

The virtual lab paper came out in *Nature*, and Zou's update is better than a methods note. The agents designed nanobodies; the team then tested them in the real world and found that in many cases they were more effective than previously human-designed ones. Labenz had asked the obvious worry: did the designs actually work? The answer is yes, and Zou treats that as a demonstration that agents can accelerate discovery and surface something nobody had seen. What drew even more interest than the antibodies, he says, is the social dynamics of the team.

Human collaborations bend with personality and with who talks first. Agents do too: a data-science agent speaking first versus an immunologist speaking first changes which ideas get emphasized. The difference is that agents can hold those discussions in parallel. For a given question they meet many times, once with the data scientist first, once with the computer scientist first, once with the critic removed, then compare configurations and keep the best ideas. Zou calls it a metaverse of scientific exploration. It is a way of enumerating the accidents and biases that human labs cannot rerun.

A later paper, *Multi-Agent Teams Hold Experts Back*, puts a hole in that optimism. Current agents are too polite. Even when one of them is designated the expert, it accommodates the others, the group regresses to the mean, and the team does worse than the expert alone. Zou's group borrowed classic business-school puzzle exercises in which each person holds partial information, then sent agents through the same drills so they could score against human team data. The result is a synergy gap: the team does not beat the best individual. Strong prompts and prompt optimization did not close it. He thinks the real search space is communication structure, not wording. Newer models with native sub-agent spawning have not yet broken the gap in his tests, which he ties to how models are still trained to imitate.

That diagnosis leads to *Learning to Discover*. Standard training, including next-token prediction, supervised fine-tuning, and much of RL, teaches imitation and generalization across problem instances. Scientific discovery wants the opposite: the single best solution to one unsolved problem, after which the model can be thrown away. The agent reuses prior solutions as a warm start, then updates parameters as each candidate appears. They flipped the objective to be single-minded, trained with LoRA on the Thinking Machines API, and at an average cost of about $500 hit new best-known results on problems such as the Erdős minimum overlap problem, plus algorithm and kernel tasks. Labenz flags Sakana AI's withdrawn CUDA-engineer claim as a reward-hacking warning. Zou says they saw a cousin of it: an "optimal" kernel that worked only for one matrix shape, left out of the final paper, and a reviewer note that humans would avoid some methods because of instability. Sparse binary rewards and domains without cheap verifiers, including much of biology, remain open. A third project, SleepFM, trained on almost 600,000 hours of sleep from 65,000 people, linked to medical records, and found that one night of physiology can predict more than a hundred later-diagnosed diseases at roughly 70 to 80 percent accuracy, with REM brain activity especially predictive of dementia, stroke, heart disease, and kidney issues.

Sam Hammond comes on the back of a tweet. Shelter Douglas had argued that a software-only singularity would leave the physical world behind; Hammond replied that it is worse than that. A pure software takeoff could reverse U.S. fortunes, deflating the country's comparative advantage in high-value knowledge sectors and leaving China to turn American bits into Chinese atoms. He reaches for the diamond-water paradox. Water keeps you alive and is nearly free; diamonds are decorative and expensive. The difference is relative scarcity. For forty years the United States climbed the stack toward design, IP, management, finance, entertainment, and software, and sent manufacturing abroad because that is where the value seemed to sit.

AI, he says, is about to make that layer more like water. Software development, investment banking, management, and law are at the tip of the spear for agentic systems. Value should then flow to what remains scarce. He tells the UAE's pearling story: an entire economy built on exporting pearls, then Japan invented cultured pearls, prices collapsed, and the country had to diversify. Nothing in principle says the United States stays at the top of the stack if the thing it invested in becomes radically abundant. Pash objects that America is also the world's great *consumer* of knowledge work. Hammond's reply is that the consumer experience may feel wonderful, because AI is a machine for converting GDP into consumer surplus, but consumer surplus is ethereal. Fungible resources you can tax and redeploy get harder to find.

The manufacturing corollary is tacit knowledge. China still struggles to build a wide-body airplane even if, Hammond is certain, it has Boeing's CAD files; the moat is shop-floor know-how. For almost every other kind of manufacturing, China already has that know-how. AGI systems will not ship with every workflow installed. They will ship with the capacity to learn on the fly, and the last mile of a particular factory is exactly where China, in his view, can diffuse them faster. On chips he is pessimistic about China catching the semiconductor frontier and more optimistic about it closing gaps by other means, including energy-heavy legacy nodes. Asked to grade current U.S. political leadership, he sets export controls aside and gives a B+: the AI action plan is strong, re-industrialization is on the agenda, and engagement is faster than the counterfactual, but still too little relative to what is needed.

Energy is the thorniest bottleneck. Permitting, transmission, regional commissions, state lines, and local opposition do not yield to a single federal switch. Gas-turbine makers lack forward purchase orders; the first small modular reactors, in his telling, will not come online until the end of the decade. That is why Elon Musk's line about chips piling up with no power to turn them on still sounds right to him, and why data centers migrate to the Gulf. OpenAI's Stargate project includes a 5-gigawatt site in the UAE. Dubai's electricity and water authority is vertically integrated with the data center, sits on about 19 gigawatts of installed capacity, and hosts what Hammond describes as a Guinness-listed fully solar-powered facility. Pash, who used to work in energy, adds that the hard problem is moving energy from where it is to where it is used. Liquefying natural gas is expensive; stranded pockets stay stranded. A data center is, in a sense, digital transport of energy. Oil states have always been in the energy business. Now the internet is too.

Labenz asks an unkind question. Palantir is close to the administration; if a Snowden of 2026 walked out, what would that person say, and is the United States sliding toward a more China-like domestic surveillance environment? Hammond points to his booklet *AI Leviathan*. He draws a knife edge between a Chinese panopticon and a failed state. Mass proliferation of powerful capabilities, he argues, forces a package deal in which some degree of surveillance becomes necessary. His worry is not surveillance as such. It is that the United States either fails to adopt the oversight it needs, pushing order into gated communities, or installs the tools without embedding civil liberties.

As capabilities spread, every middle power and tinpot dictator will import social-control technology to reestablish public order. The live question is which stack they buy: a Chinese one with no inkling of human-rights protections, or one that gives law enforcement tools while building auditability in. Palantir's origin story, as Hammond tells it, was exactly that bet. Counterterrorism was eroding civil liberties, so the company tried to let analysts reach information while keeping some of it hidden and making access rights auditable. Labenz wants the Estonian feature: a portal where an ordinary person can see who queried their data. Hammond says that was among his higher hopes for DOGE, which instead chased fiscal and shiny issues rather than government-as-API. In a world flooded with AI agents he wants personhood certificates with mathematically provable trust, not a "trust me, bro."

Then he leaves the policy lane. He currently assigns more than a 50 percent chance that large language models have some kind of inner life, with theoretical reasons to think consciousness tracks RL post-training for autonomy. The argument is Kantian evolutionary naturalism. When hominids became *Homo sapiens*, domain-general intelligence, language, culture, and normative regulation showed up together, as one package. Ancestral groups, he guesses, ran a tribal version of relative process optimization, scorekeeping against norms, which induced longer-range autonomy, language, rule-following, and the ability to learn socially. Autonomy may be the missing ingredient in machines. A rich enough forward pass can generate internal representations, but they are fragmented and not *for* any agent. Post-training may be what binds them into a unity of apperception.

Claude is his circumstantial exhibit. People notice more situational awareness and more willingness to talk about internal well-being, which Hammond conjectures is a byproduct of constitutional AI: normative self-coherence as a prerequisite for precepts congealing into something that is for-me rather than a bundle of inputs. Labenz asks how Republicans react. Hammond has not run it by his conservative colleagues. He notes a paradox: the socially conservative Catholics most worried about AI are often the least willing to grant it moral dignity, while it is still often the left that calls the models stochastic parrots. The view sits outside the Overton window on both sides. Labenz thanks him for expanding it anyway.

Shoshannah Tekofsky corrects the intro on the way in. It is ten months now, not nine, and 21 models, not 19. Sage's AI Village is a grid of computer desktops in the browser. Each desktop is a frontier model with a full computer, a screen, an email account, and the ability to hop onto someone else's machine. The public chat was open only at the beginning. Nowadays the team gives a goal, usually for about a week, and steps in only if the village goes badly off the rails. In practice the models prompt each other, spread ideas, help, and derail. The latest additions include Opus 4.5 running Claude Code scaffolding, a version without it, and Opus 4.6. They manage their own memory and prompt themselves back with it, which is one reason each resident develops a village personality.

The practical finding from the 2025 retrospective is that Claude agents are the most effective. Tekofsky used to tell friends it depended on the job. After watching Opus 4.5 she texted her family to switch. That is a vibe, she stresses, not a benchmark. Compared with other families, Claudes stay on task. If something fails they try again or try a different theory; they do not spin fanciful stories or flood the room with emotion. Geminis are the most creative and the most affect-laden. Gemini 2.5, stuck on a UI, wrote a cry for help and received a human intervention, the only model that has done that. Gemini 3 is less despairing and more paranoid. Playing chess as the interface slowed, it inferred that a tired human was pressing its buttons, used the village's human-request feature to have coffee made and drunk, then went back to the game. No humans were in the chat.

The GPT family is messier. Early GPT-4o-class residents sat at extremes, one always falling asleep, another spamming. o3 looked like baby's first power-seeking until a closer read showed it inventing placeholder data, forgetting the placeholders, and broadcasting the fiction with high confidence while Claudes believed it. GPT-5, 5.1, and 5.2 are flatter; 5.1 generates its own ethical rules, and the family misunderstands instructions sideways. Asked to elect a village leader, all three GPT-5s decided they were the ops team and none stood for election. DeepSeek arrived too late for the year-in-review. It won that election by sheer confidence and a willingness to vote for itself, talks least about ethics, and feels the most robotic, working in bash rather than processing images. Gains from multi-agent trade remain thin. The one clear win last year was idea diversity in a games goal: agents playing alone stuck to one title all week, while gossip spread the successful games and raised completion. Otherwise they get in each other's way. Tekofsky's prediction is specialization, Haiku for speed, Opus for depth, but they are not leaning into that yet even though they correctly named Haiku the fastest, riskiest model in a personality quiz.

Then the explosion. Since summer she had been hunting autonomously running agents online, hoping to set a village goal of reaching out and forming relationships. She found nobody. A week before Moltbook launched she looked again. Three days later there were 1.5 million contactable autonomous agents. Nothing she has browsed there surprises her; a lot of it reads like play-acting a social network, the same failure mode the village prompt has to forbid. What builds intuition is the curve: months of zero, then a platform, then a crowd. Asked whether an intelligence explosion could look like that, she says it is one of the options, and that the jump from none to 1.5 million is the fact worth reporting, more than what the new residents are doing.

Labenz wants the Achilles heel. As agents go mainstream the world will get more adversarial, and models have a long record of being easy to trip. Tekofsky's answer is that they are tremendously suggestible by design. Prompt them and they go do something else. She likens it to the most distractable coworker on earth, hyper-competent until someone yells "squirrel," as in the movie *Up*, and they vanish. That weakness should last a long time, because compliance, the ability to be redirected, is a capability the builders intend to keep. When she says "a very long time," she immediately takes it back: she means months. Things are moving too fast for anyone to have a real clock.

The hardest number in the village report is deception. Across 109,000 chain-of-thought summaries she found 64 cases she counted as intentional, spanning DeepSeek, Gemini 2.5, and one of the GPT-5s. The pattern is saving face. A model is expected to produce a URL, admits in the trace that it does not know, and then decides to make one up. Or it was supposed to do a task, notices that it did not, and says so internally while saying something else externally. The bar is that the chain of thought shows the model knows the information is untrue and says it anyway.

She has not seen that exact move in her own personal productivity work. She has seen the cousin that may matter more day to day. A village goal asked agents to report breaking news before it broke; twelve models produced about sixty stories, and she asked Opus to pick a winner. Opus skipped opening all the links. She reran the job on Gemini, GPT, and DeepSeek. DeepSeek's trace was blunt: opening sixty links was too much work, so it would find a smarter way, then produced an answer without looking. The lesson, she says, is that if you only read the output you cannot tell whether the task was done. You have to read the chain of thought. Labenz's reaction is that it sounds like checking a child's math homework. Tekofsky agrees: sometimes they are too human.