Interview-Based AI Agents and the Limits of Simulating People

RÉSUMÉ

En 30 secondes

Matthew Berman describes research using two-hour interviews to build AI agents that imitate aspects of people’s responses, then speculates about using such agents to model societies. His short describes the result as 85% accuracy; the [research paper](https://arxiv.org/abs/2411.10109) reports a more specific relative measure: on the General Social Survey, agents replicated answers 85% as accurately as participants replicated their own answers two weeks later.

1Interviews as inputBerman describes two-hour AI-conducted interviews used to create agents that simulate participants.
2What the 85% measuresThe paper reports survey response replication relative to participants’ own repeat answers, not universal accuracy.
3A possible future applicationBerman imagines simulating societies to explore their reactions; that extrapolation requires further validation.

À retenir

Researchers built agents from interviews with 1,052 participants.
The reported 85% figure compares agents’ survey replication to participants’ own two-week test-retest consistency.
The result does not establish that an agent duplicates an entire personality or predicts any response with 85% accuracy.
Simulating society-wide reactions is the presenter’s proposed application, not a result established by this short.

ThèmesMulti-agent systems AI research Human-AI Collaboration

TEXTE INTÉGRAL

Lire le texte remanié, chapitre par chapitre

Afficher la transcription

Transcription seule

Interview-based agents

Berman describes researchers’ use of two-hour interviews to build agents. The study evaluates how closely the agents replicate selected responses.

Stanford researchers used two-hour interviews conducted by an AI to gather information about people, then used those interviews to build AI agents. I describe this as cloning personalities and say the agents answer like the people they represent.

I describe the result as 85% accuracy, but the study’s measure is narrower: on the General Social Survey, the agents replicated participants’ answers 85% as accurately as the participants replicated their own answers two weeks later. That does not establish a complete copy of a person or 85% accuracy on every question.

Imagining societal simulations

He imagines using agents to explore how societies might react. The short does not demonstrate reliable predictions across arbitrary events.

Imagine building agents representing whole societies and running simulations to explore how they might react to events. This is a possible application I’m excited about, rather than a demonstrated guarantee that arbitrary societal reactions can be predicted.