How Generative Agents Populate a Simulated Town

I am excited about a project from Google and Stanford that brings artificial intelligence into a sandbox world. Its characters can form memories, reflect on experiences and use them to inform future behavior. I see the potential for non-player characters to have lives of their own instead of waiting for a player to interact with them.

The paper is called “Generative Agents: Interactive Simulacra of Human Behavior.” It describes an architecture for characters that interact with their surroundings and one another with little user input, using a language model. The researchers give each agent roughly a paragraph describing its identity, and the simulation develops from that starting point.

The examples include agents waking up, cooking breakfast, going to work, painting or writing. They notice one another, form opinions and initiate conversations. The especially interesting part is that they remember and reflect on earlier experiences while planning what to do next.

The architecture records experiences in natural language, synthesizes higher-level reflections over time and retrieves memories to help plan behavior. That suggests a different kind of game character from a static NPC, though this is a research prototype rather than a finished game.

One example starts with an agent given the intention to organize a Valentine’s Day party. Over the next two simulated days, agents spread invitations, make acquaintances, ask each other on dates and coordinate to arrive together. Berman is struck by how much social activity follows from a relatively simple initial direction.

The challenge is to handle memories that keep growing as interactions and events occur, while social dynamics cascade across agents. The system needs to retrieve relevant experiences, draw higher-level inferences and use them to produce reactions that fit both the immediate situation and a longer behavioral arc.

Games are the application that most excites me, but the paper also mentions role-playing difficult conversations, such as interview preparation, and prototyping social platforms. Simulated agents could interact with a proposed platform before it is released to people, potentially exposing problems for designers to investigate.

The researchers built a sandbox town called Smallville, with a park, a co-living space, a bar, a café and houses containing familiar rooms. Twenty-five agents inhabit it, each represented by a simple sprite. Each begins with a paragraph describing an identity, occupation and relationships as seed memories.

One resident, John Lin, is described as a pharmacy shopkeeper who likes helping customers. His seed description includes his wife, a college professor, his student son and his neighbors, including an acquaintance with whom he discusses local politics. Those details establish a starting context for interactions.

The agents communicate in natural language. In one example Isabella discusses an election with Tom, who expresses a negative view of candidate Sam Moore. These conversations are generated between characters inside the simulation, rather than being presented as fixed NPC dialogue.

A user running the simulation can intervene by talking to an agent or issuing a directive as its inner voice. For example, a prompt expressing an inspiration to paint can influence an agent’s subsequent plan. Agents can also develop routines, such as waking up, washing, eating breakfast, catching up with friends and beginning work.

The architecture diagram begins with perception: events are recorded in a memory stream. The agent retrieves memories and uses them in reflection or planning before deciding on its next action. The stream records observations over time, such as a room’s objects being idle or Isabella cleaning the kitchen.

When asked what she is looking forward to, Isabella can retrieve the fact that she is planning a Valentine’s Day party at Hobbs Café. The retrieval mechanism weighs recency, importance and relevance. The paper’s diagram traces how Isabella tells other agents, who in turn pass the invitation on; one chain goes from Isabella to Sam to Jennifer.

The researchers also provide a demonstration. The run shown here is prerecorded, not a live simulation, but it lets viewers watch the characters move through part of Smallville and inspect individual agents.

Selecting Francisco Lopez shows his location and current activity, having lunch, along with whether he is in a conversation. Selecting Sam Moore shows a conversation with LaToya. Berman uses the interface to illustrate how the 25 agents carry on activities and interactions within the simulated world.

I can imagine ideas like this changing games such as The Sims, RimWorld, Skyrim or GTA. Instead of characters standing by with static dialogue, a future game could incorporate agents whose activities affect a changing world alongside the player’s actions. That is an imagined application of the research, not a demonstration that those games already work this way.