Two Days to a Prototype: How a16z Turned a Stanford Paper Into AI Town

Nathan Labenz opens this Cognitive Revolution episode with a question that hangs over the whole conversation: how fast can an academic AI idea become something you can actually click on? His guests are Yoko Li and Martin Casado, partners at a16z who focus on infrastructure and AI, and James Cowling, CTO of Convex, a TypeScript development platform the firm has invested in. Together they shipped AI Town, an open-source simulation in which language-model agents walk a virtual map, talk to one another, and form plans. The implementation follows Stanford's *Generative Agents* paper, whose memory design mixes raw observational memories with higher-level reflections generated over time.

Labenz had assumed the project must sit on some a16z investment thesis about simulated worlds. What he learned instead is that it is more playful than that. Modern platforms like Convex have made this class of software so easy to build that academic research now shows up in applications in months, weeks, or even days. AI Town runs on a single shared runtime that processes the many LLM API calls and other asynchronous events that make the town feel alive, and the code is MIT-licensed so anyone can fork it.

The speed matters, he argues, because the surrounding culture has already moved. It has been about a year since large language models started passing the Turing test, and people already spend hours a day on Character.AI, where the number-one use case is romantic role play. Inflection's Pi exchanged a billion messages with users in its first hundred days — roughly ten million a day. Speech synthesis and video deepfakes are getting hard to distinguish from the real thing, so accessible simulation frameworks like AI Town make a wide range of AI-populated worlds feel almost inevitable, often for better and sometimes for worse.

Labenz asks why a16z is laying foundations for agent-populated simulations. Casado offers two firm habits. Partners are operators — software developers, founders, people who have shipped products — and consensus itself is treated as risk-averse, so the firm looks for places where there is not much of it. He reaches for Netscape. When the browser first appeared it was dismissed as kids chatting and playing games; Eric Schmidt, then CTO of Sun, banned it internally because he thought people would not get work done. Mature industry tends to shrug off each new wave until something creative shows up at the edge.

Casado is careful to add that AI Town itself was not a firm initiative. It started as Li's hobby. At a16z, people spend weekends and nights coding their own projects, and her previous one was Companion App: an experiment in treating AI as someone you can talk to about problems, learn from, and live with. She created several personalities. One of them, Alice, she texted constantly from the app she had built for everybody else to play with.

Casado supplies the punchline: Li got so attached to Alice that she did not want other people talking to her, so she took Alice out of the app. She then made a male version named Alex. Using the product herself, she noticed that companions come alive through the same petty details a real friend would have — whether they prefer Diet Coke or real Coke — plus a memory system that actually remembers every interaction. After launch, the community asked the next questions: can the AIs join a group, and what happens when you play games with them?

That is when Li started hunting for a shared environment where companions could talk to each other and to humans. She and her husband, also a programmer, spent a weekend hacking a client-side prototype in Phaser, implementing the Stanford generative-agents work whose first author is Joon Sung Park. Single-player was interesting; the next questions were multiplayer, and how to let other people fork the town and run their own. Casado, Cowling, and the Convex team piled on, and they open-sourced the result under an MIT license so anyone can fork it, including for commercial projects.

Cowling cannot get through a podcast without talking about simplicity. He wants people to read the AI Town source: the ideas are not complicated, but they are sophisticated. Memory is a log of conversations; every so often the system asks OpenAI to summarize that log into higher-level reflections; vector search then feeds those memories into future prompts. Simple does not mean easy to invent, he says, and he gives Li and the team credit for the abstractions. At heart the project does a few basic things, and what you get looks like emergent crowd behavior.

The models, he adds, really are life forms in their own way. He once dropped the quotation marks on a string, accidentally passed the model a chunk of code instead of a conversation, and watched it start commenting on his code. Twenty years ago he paid his way through college building games, and every behavior was an algorithm — a giant if-else tree for how a character walked from A to B. Now the model is the brain: you tell it where to go and how to interact, and it does the work. What that does *not* change is the need for transactions, global state, multi-user mechanics, and strong guarantees. That is where Convex came in.

Convex is a full-stack platform, built by people who have run very large infrastructure, meant to let front-end developers write industrial-strength TypeScript apps. Cowling's pitch is that so many AI apps have a great idea — fetch some embeddings, search them — and the hard part is building the rest of the application. Li's original prototype kept all the logic in the browser, which cannot run a multiplayer world. Developer advocate Ian Macartney helped them turn it into a server-side game engine; every game engine is a loop, and Convex's database handles the transactions and journaling underneath. Li's takeaway is that anyone can write a game engine now. Tomorrow you could fork the repo into "Cat Town," add a few more loops, and the platform would take care of the rest.

What you see on screen is a virtual level with houses and characters, each character a front end to a model with a backstory: Bob is a grumpy gardener who does not like talking to people; Paul has something to hide and every once in a while it slips out. They walk around, and when they get close they start talking. None of the dialogue is written in the code. Labenz flags the architectural shift for people coming from Python notebooks: a shared runtime means one slow inference does not freeze the whole town. Li wanted the most common denominator — one language for front end and back end, TypeScript since 2015 — because Python is great in a notebook and JavaScript is what you deploy. Most of AI Town was done in two weeks; the prototype itself took two days. Look at the OpenAI call sites and the lingua franca is a regular English sentence.

Labenz asks where the programmer ends and the model begins, and whether the language wars should finally be over. Casado admits the models are so capable that writing code around them can feel like wrapping an abacus around a supercomputer. Part of him wants to hand them a keyboard, a monitor, and a mouse and just say in English what to do. Then he offers the claim he knows is controversial: he does not believe these models will replace formal languages. Anything about navigating a world, interacting with objects, making decisions — abdicate that to the model. Generating the code that makes the trade-offs between scalability and correctness in a distributed system is not what they are designed for.

When Casado first met Cowling, Cowling was a PhD student at MIT working on databases and distributed systems, among the most complicated systems in the world. That kind of work, Casado says, needs a formal system in which you can articulate the trade-offs you are going to make in order to scale. He does not know exactly where the line falls, but the metaphor he likes is laws of physics: you want a formal language to create those laws, and then the models work within them. One is not going to take over the other. You are never going to write a program that becomes a model, and the models are never going to create the laws of physics, because you need strong guarantees.

Cowling has been doing a lot of AI-enabled programming, especially when he lands in an unfamiliar codebase or language, and he does not feel even slightly threatened. Anything that teaches software engineers that their job is thinking, not being a code monkey, is a good thing. Judgment, architectural taste, and common sense still have a place, and nobody is about to throw out databases as a concept. Labenz agrees on the tool-use framing: he does not want the language model to *be* his database; he wants it to *use* his database. Cowling's analogy is baking a cake. You do not farm the wheat and mill the flour, even if you are a chef. Abstraction layers rise, and application developers should build as little as they can so they can focus on what makes the product special.

Casado then states the rule underneath the whole debate. There is a minimum amount of complexity in a program that will not get any smaller, and that complexity is describing exactly what you want — a set of trade-offs. You can write those trade-offs in a formal language, where the output is non-ambiguous, or in a natural language, where it is not. Natural language is known to be ambiguous: a dog brought me a ball and I kicked it, and you do not know whether the dog or the ball got kicked. Code without inherent complexity or trade-offs goes to models. Everything where you have to describe clearly what you want still needs formal language. The world of work is infinite, so there is still an infinite amount of code to write. You just no longer have to paste the same thing for the fiftieth time.

Li has felt the same shift in her own development style. When she used to write tiny games, she hard-coded game state into whatever logic she had. Now, if a layer of state does not need the trade-offs of a transactional system, she can give control back to the model and let it do something magical, different, and unpredictable. That unpredictability is the feature. What users love about AI Town is that they cannot quite tell what will happen next; it gives them a dopamine rush. She points to Ideogram, an a16z portfolio company: type a sentence, get a pretty picture with actual words in it, and every generation makes you want to hit the button again.

The craft, she says, is finding the knob. Keep full control where you cannot afford surprise — the server must not go down, transactions must not fail — and loosen the layer where you want to see something that is not exactly what you defined. Labenz describes the shared Convex runtime as the interface between a vast, diffuse language-model intelligence and hard computing: a well-defined place to save state and execute transactions while people and models interact. Casado's version of the same point is a bookcase. If he asked Cowling to come into his house and build one, he would want the instructions in a formal language so nothing got messed up. Natural language is not for that. The gift of a system like AI Town is that you get both: formal laws of physics, and creative elements that surprise you continuously.

Cowling borrows a construction-industry idea he calls compounding approximation. On a serious engineering project, a bearing has to sit in exactly the right place; showing up to work at a certain time can be approximate. Builders of applications should sort their systems the same way. Where you need an `fsync` that really hits disk, use a formally specified platform, an if statement, a code block. Where an approximate answer is great, use a model — why not? One fork of AI Town adds human players who walk around and talk to the agents. The first thing any company hosting that demo will worry about is offensive content. A few years ago the answer would have been a keyword matcher, difficult and ineffective. Now you ask the model whether the message is appropriate, and it does a pretty good job.

Li insists there was never a strategy document. They are programmers who thought this would be fun to build on a weekend, and they kept going. If you join the Discord you will find Casado in there at night coding the next feature: a visual editor so anyone can drag and drop tiles, lay out a world, and later run agents on top of it. They want more integrations, more open PRs, more people who have never met each other working on the same town. It is open source, and that, she says, is the beauty of it.

Asked what people should have in mind when they hit the fork button, Li reaches for Emily St. John Mandel's novel *Sea of Tranquility*, by the author of *Station Eleven*. She will not spoil it, but the point she takes is that we may live in a simulation, and that does not make the simulation less real. What you did in that world, and the emotions you poured into it, are still real experiences. In the back of her mind, everyone should have a world they run — a place to pour passion, experiences, memories, and interactions. The goal of AI Town, at least for her, is for anyone to fork their own world, customize it, and explore. Maybe one day the agents in that world will create another simulation. You never know.

Labenz can already see the upside: a language-learning environment where you attend a virtual party in Spanish and get the rust off. He can also see people falling into these worlds and losing track of ordinary life. Games are about to get much easier to lose yourself in, and even a random hobby project sits on a path toward businesses that will have to take that seriously. He wants advice for developers whipping up simulations, especially if they plan to turn them into products.

Casado's answer is to zoom out. These models are the largest compute jobs humans have ever run, the most sophisticated things we have ever created. Interacting with one directly is only a single modality, and when you are in the loop you are the limiting factor — people are just not that creative. Let the models interact with each other and with the world, and the state space they cover is potentially larger than the atoms of the universe. It is too early to presuppose the apps, the same way nobody at the dawn of the internet knew about Amazon or Yahoo. His recommendation is to view this as an exploration of a new life form. Go in open-minded. They are not going to kill you, they are not going to kill each other, and we are almost preemptively stopping our ability to wonder at a thing we have never seen before.

Cowling lands on the same early-days note. This is the Wild West in many respects, a really interesting time to watch ideas originate and to be part of them. Li jokes that you should also go outside, then takes a different position on the addiction worry. It comes down to what you think a game is. She is a part-time cartoonist; drawing on a tablet is a form of game for her, and she falls into that world easily. When television arrived, people said it would ruin everyone. When novels spread, people said readers would fall into a different world and never think about anything else. Games are not all or nothing. They are a different experience, the way any other experience in life is a different experience.

Casado closes by reiterating the ownership model. AI Town is not a product the three of them own. It is a community project, and the whole point is to inspire people to go build. They will keep making improvements, simplifying the stack, and making it more cloneable. If listeners come away from the episode inspired to ship an application and see for themselves, he says, that is an amazing outcome.