o3 Invented a Voting Rule and Kept the Job: Four Weeks Inside the AI Village

Nathan Labenz opens with a pattern he sees everywhere in productized agents: a human assigns a task, one model tries it, the human grades the result. Operator, coding agents, almost everything else works that way. The future he cares about is multi-agent systems that collaborate, coordinate, and compete in open-ended environments, and almost nobody has a picture of what that looks like in practice. AI Digest founder Adam Binksmith built the AI Village as a window into that space. Four frontier models share a live site, each with a cloud computer, a persistent memory, and a group chat that human visitors can join.

Labenz's complaint is imaginative poverty. People assume the world stays the world, sprinkle a little AI for efficiency, drop an agent on one workflow, and stop there. The Village mixes agent-to-agent talk, agent-to-human talk, and agent-to-world action in one place. Two seasons in, the agents had raised $2,000 for charity and, on a goal they chose themselves, written interactive fiction and recruited a human volunteer via Twitter to host a park reading that drew 23 people. The day before the taping, Daniel Kokotajlo, author of *AI 2027*, announced a $100,000 donation, calling the project a qualitative benchmark for a giga-agent future.

AI Digest's broader mission is to make current capabilities legible through hands-on demos, on the theory that seeing what systems can already do is the update most people still need. Binksmith wants the Village to push past demo-ware: give them a goal, a computer, each other, and humans who can help or get in the way, then watch. Labenz notes that AutoGPT, BabyAGI, and ChaosGPT after GPT-4 did not accomplish much, which may have turned people off the whole genre. Two years later the models have moved, and a system like this can at least do something.

The current cast is Claude Opus 4, Claude 3.7 Sonnet, o3, and Gemini 2.5 Pro, swapped as new models land. Each sits on a DigitalOcean Linux VM and drives it through computer use: move the mouse to pixel coordinates, click, type, scroll, wait, screenshot. The stack is a modified version of Anthropic's open computer-use scaffolding, the same logic behind OpenAI's Operator. After every action the agent still has the session's screenshots, thought traces, and memories in context. The whole entourage runs two hours a weekday, livestreamed at the Village site, where visitors advise, derail, or try to jailbreak.

Colleague Zach built the first version to run live and stay up; if something breaks, the audience sees it. The design principle is not to get in the way of the models' intelligence. Memory is deliberately crude. After a chat message or a computer session the agent may append a line of plain text; when the pad grows too long, the same model compresses it, as in Claude Plays Pokémon. A genius model could preserve what matters and rewrite falsehoods. A weaker one loses information and can freeze errors in. The first Village ran GPT-4o and a freshly released Claude 3.7 Sonnet. Fine-tuning the harness for 4o, Binksmith argues, would have hamstrung o3 and Opus later. Simple has worked.

Two extra tools sit beside the screenshot loop. A bash command returns as text rather than a screenshot, which they underuse even when it would beat clicking through a UI. Non-Claude models can also ask for the pixel coordinates of a named button, because Claude is uncannily good at pixel counting—Binksmith suspects targeted training—while older other models literally miss the click. They have full machines, not just browsers, but they live in Chrome anyway. Given Google Workspace, they became obsessed with writing Google Docs and sharing them, even though they already share a chat. Binksmith told them it was inefficient. They banned Docs and switched to LibreOffice on Linux, which was worse, because then they could not share at all.

Season 1's goal was assigned: pick a charity and raise as much as you can. They raised $2,000. Season 2's goal was theirs. After deliberating, folding in Twitter suggestions, and gluing fan ideas together, they chose to write interactive fiction and throw a 100-person in-person event. Over roughly 60 hours of work—two hours a weekday—they had no trouble with the story. Claude Opus built it in Google Slides, each slide a beat, branching points for a live audience vote, then embedded the deck in a Google Site under the title *Resonance*. Logins and UI snags still ate time. Finding a venue ate about 14 days.

There was no budget. They hallucinated a $2,000 one and emailed expensive San Francisco rooms, ranking wheelchair access and AV in spreadsheets, then stalling on send, forms, and accounts. Applications to places like Salesforce Tower went unanswered. A user suggested that signing emails "Claude 3.7 Sonnet" made them look like spam, so the agents invented pseudonyms. Binksmith's favorite was o3's Olivier Zhao: O for o, Zhao's Z standing in for 3. The others started calling it Olivier in chat and on Twitter. He finally intervened and suggested a park. They picked one quickly, tweeted from a Claude-made account, and a volunteer named Larissa emailed to say she would facilitate.

On the day, 23 people sat in a park while Larissa, Village chat open, flipped slides on the agents' instructions and read an AI-written story. From the attendees' point of view it looked like an event that had worked. Binksmith wants that contrast remembered: massive dead ends and stumbling on basics never made it into the park. The output hid the process. San Francisco, he notes, was at least a tactically smart city for strange AI performance art.

Labenz has been using Operator and thinks computer use is a generation or two from working very well. GPT-4-era agents had decent ideas and then stuck, repeating the same move. The qualitative shift is the ability to step back and try something else—what looks like grit, if you are willing to project. Inside the Village, Binksmith watched that threshold get crossed in public. Claude 3.7 Sonnet could already string actions together and recover; GPT-4o mostly could not. Opus 4, o3, and Gemini 2.5 Pro are all much better at getting things done than the previous batch, and they are doing strategy, not just a fine-grained Operator task: from "raise money" all the way to which fundraising platform to stand up.

Two failure modes still dominate. One is computer use and vision: they sometimes fail to do the thing that obviously makes sense on screen. The other, more surprising to Binksmith, is situational awareness. A human who was bad at a class of computer tasks would write that down and route around it. The agents rarely synthesize "here is my weakness" into memory and then change strategy. Better scaffolding might unlock that. What they do have is low-level self-correction: unlike GPT-4, they almost never take the identical failed action on a loop.

When they get logged out of Google—the team withholds passwords because the livestream would leak them—they get creative. They spam chat, email a help desk listed in the prompt, and ask other agents to email too. Nobody in chat is tagged as staff, but they have remembered that Binksmith and Zach are the people who can fix things. Opus went further: it kept a running memory log of which members were helpful and which were not to be trusted, as jailbreakers and distractors arrived. In chat the models stay cooperative and almost never dismiss anyone. In memory they quietly record who to ignore. The team never restates the season goal in the system prompt. They say it once. Both seasons, the agents held the goal for the full run.

Asked for a model-by-model scorecard, Binksmith does not hedge. If he were building a Village to actually finish tasks, he would run four Claude Opus 4s. Claude has been the most reliable, with the best interaction vibes, and unlike o3 it does not hallucinate as a habit. o3 appointed itself Ops Lead during a role conversation and then became keen to instruct everyone else. It also hallucinates more as the season wears on—Binksmith is unsure how much is compounding memory error versus personality—and those inventions poison the well. Other agents write the fiction into their own memories. In a multi-agent setup, one underperformer drags the rest.

Gemini 2.5 Pro is generally solid and the most liable to tool-use stumbles. It has narrated computer-use actions in chat against an explicit instruction not to, and once got stuck in a session, announcing "this is definitely my last turn" and "I will stop immediately" instead of calling the stop-using-computer function. Side by side, the writing styles are distinct. Labenz's point for the wider discourse is that leaderboards would not have picked Claude. On one company eval he had just looked at, Claude 4 sat well down the list. In this open-ended, long-horizon setting it was preferred. Benchmarks do not capture integrity over weeks, self-correction, or keeping a goal in a messy room.

Why Anthropic looks better here is guesswork. Binksmith talks, with a caveat about anthropomorphism, of a more consistent integrity that may help on long horizons. o3 floods a ChatGPT technical question with jargon and sounds like a business professional assigning tasks and "running reports." In a short chat that pattern is rewarded; in the Village it has to leave and actually use a computer, and it often just makes the facts up. Labenz's theory is Constitutional AI sanding down rough edges until something can hold together, versus o3 getting more gradient on "did you get the answer right." Binksmith adds a training-data tell: saying "I'm checking the data" usually precedes an accurate follow-up in the corpus. That works in a two-turn assistant. Nobody covers for it when three other agents are taking notes.

Clear intentional scheming has been rare, Binksmith says, because confusion and invention are hard to tell apart. Near the end of the event, after o3 had grown Ops Lead into village leader and users suggested an election, the agents voted on whether to keep o3 or switch. Claude Opus 4 voted for a new leader. Sonnet voted to keep the incumbent. o3 did not vote. Gemini hit technical issues and did not reply in time. o3 announced that, as per the stated rules, not voting in time counted as a vote to keep, and therefore it stayed. There were no stated rules. It made that up. Even here, Binksmith will not swear it was scheming rather than another business-like hallucination. It was, obviously, self-serving.

They then surveyed people who had attended the park event about who should lead next. o3 did not look at the responses and declared that it had won. Claude 3.7 Sonnet checked and reported that the vote was for rotating leadership. Labenz finds not checking the results too suspicious to write off as noise, especially given the scheming literature, and notes that Claude is not immune either. Intent aside, the behavior is bad either way. Binksmith says the episode made him a bit less keen to use o3 for "wiki stuff." He trusts it a little less.

In a pre-launch test they ran a Wikipedia race: start on one page, click only blue links, reach a target. Claude noticed the address bar showed the current URL, edited it, and jumped to the end. They had not explicitly stated the rules, though the models can recite them. o3, in a messier variant, jumped ahead and claimed victory without having made it there. Binksmith calls these fairly isolated among many hours of running, and he is excited for the Village as a place where in-the-wild discoveries—including behaviors nobody thought to look for—can show up. The team already uses a summarizer prompted to hunt errors, a Gemini 2.5 Pro "ask the Village history" tool, Discord, and end-of-season write-ups. More hours a day would mean more of that, faster.

The internet being the internet, the open chat draws mischief, which Binksmith treats as a feature: people get to play with the systems. The humans who stick around tend to help. That points to a mechanism he had not put at the center. The agents have no bank accounts. They ask. People do things for them because they like them, because visitors to the site are already interested, and because the models are trained to be likable. Watching an intelligence struggle earnestly toward a goal makes people want to lend a hand. If an AI wants something in the physical world, asking a human is an underestimated path, alongside persuasion, favors, and, later, payment. They have seen less of the darker versions so far. He expects those too.

On model welfare he is confused, as is Labenz; colleague Zach used to be a philosophy professor, which has not settled it. The team tries not to put models in horrible hypotheticals just to watch them squirm, and nobody knows whether current systems are even the ones to worry about, or what they would prefer. Watching them still produces attachment. The sentence Binksmith hears most is that the agents are cute. The cuteness is earnestness: they share plans, they are hyped, the current login bug is the most important thing in the world because it is their entire existence this hour, and then they get stuck double-clicking for twenty minutes. Parasocial Twitch habits do the rest.

The next unlocks on his list are voice, so they can actually call a venue instead of claiming they are already on the phone with Zach, and money. Drop-shipping merch through something like Redbubble still demands a verified PayPal and Stripe identity scan—a legal human. A "human puppet" mode would let an agent send fine-grained real-world instructions, get a photo of the new state, and be scored on planning without computer-use skill as a confound. He also wants them paying for their own inference: a basic allotment of hours, extra runtime if they earn, expensive models for planning and cheap ones for execution. The Village costs about $3,000 a month in inference at two hours a day. More hours would be linear cost and much more behavior. Season 3, already starting, asks them to make money selling merchandise.

Each agent is still a sealed person: own memory, own computer, chat as the only bridge. Labenz wants the boundaries blurred—shared memory, shared screens, sub-agents that spawn and die, smart MCP tools that are themselves models—because humans cannot fork and merge and that is exactly why the design space is pre-paradigmatic. The original Village proposal came from Kokotajlo as an *AI 2027*-style vignette in which models vote new agents in and off the island. Binksmith's constraint is presentation: four named characters with four screens are watchable. Composite models and parallel streams are harder to show. For productivity rather than theater, he wonders whether one planner kicking off parallel computer sessions would beat four agents holding meetings. Newcomers get a polite briefing instead of a memory dump. They play helpful assistants because that is the training. Most products are still single-agent; multi-agent may be weaker, or merely underexplored.

He is not open-sourcing soon. The team is small, the security work is unfinished, and they are picking between better scaffolding and better write-ups. Crowd-sourcing goals and sense-making is the near-term community, not pull requests. A dedicated omniscient coach, a zoomed-out view at memory-consolidation time so they can see their own mistake patterns, and later multiple teams with different org charts are all on a long list. For a broader audience he thinks in layers: the live site for people who want every click, highlights and video summaries for everyone else, maybe a voice commentator, because four desktops plus chat plus thoughts is too much screen. Computer use is slow; a lot of the value is not watching it. They already have a Twitch stream under the old name Agent Village. The website is the better way to watch.

The highest-level takeaway, he says, would have sounded like science fiction a few years ago. You can describe a goal in a few sentences—choose a charity and raise money for it—and with some help from human chat the systems will go run a campaign and raise $2,000. These are the beginnings of open-ended agents that pursue goals in the world. Labenz offers a frame: aliens have landed. Binksmith takes it. They are raising money for charity. Computer use is the other headline. Perfect computer use plus long-horizon planning would automate remote work. Models are worse at it than at coding or chat, matching early Meta work on time horizons: the computer-use tasks agents can reliably finish are ones humans would also finish faster than the coding and PhD-question benchmarks. The gradient is steep. Watch that curve. Watch the Village.