How GPT Live keeps speaking while an agent works

Alex, an engineer working on GPT Live, and Erika, who works with voice startups, introduce GPT Live 1 with a voice agent named Marin. They demonstrate simultaneous listening and speaking, delegation and behavior in background noise.

The speakers say the model streams audio in and out continuously rather than relying on discrete user and assistant turns. It can listen while speaking and decide when to respond.

They bring in a second voice, Colin, for a collaborative storytelling game. Colin is a separate GPT Live session with its own voice and instructions.

The conversation includes interruptions and brief reactions. Alex says the model handles turn-taking itself rather than depending on a separate voice-activity detector to decide when someone has finished.

A second voice session can also serve as a simulated user in evaluations, with a chosen voice, personality and goal. The team used that approach to rehearse the demonstration.

Erika says prompts can adjust tone, accent, laughter, backchannels and how readily the voice interrupts or waits. The right behavior depends on whether the product needs an eager partner or a patient listener.

The next demo is a drawing and guessing game. A backend creates the drawing while Marin talks with the audience and responds to guesses.

Delegation allows the voice model to continue streaming audio while a more capable model or agent works on a complex request. Results can return to the voice model when the background task finishes.

The game separates the visual interface, voice interface and backend. The voice model receives the answer and hints, letting it react immediately to guesses while the backend draws and tracks the round.

The backend can send silent context updates to the model’s thinking stream. These updates keep the voice model informed so it can answer questions without starting a new delegation each time.

A commentary stream can supply content for the model to speak, while an instructions stream can change behavior or apply guardrails. The application decides which information should be silent, spoken or instructional.

The team learned that frequent spoken updates during a long Codex task could be distracting. The speakers recommend keeping the voice informed in the background and considering what the user can already see.

With client delegation, the application receives events and runs its own backend, model or agent harness. It can use transcript and application state to prepare a request and send results back in the chosen stream.

With Responses delegation, developers select a Responses API model and configure instructions and tools; OpenAI manages the delegation and supplies conversation context. Hosted tools such as web search and image generation can also be enabled.

For a final demonstration, the team plays loud background audio while continuing to talk, change the sound and issue requests. They present this as an example of continuous interaction without waiting for silence.

The speakers suggest voice can make controls easier to reach: users can describe what they want without knowing where a setting is. They say the model continually chooses whether to speak, listen or invoke a tool.

To close, Marin delegates the generation of a grid of images. The team says application designers must decide what the backend handles, what the interface shows and what the voice model needs to know or say.