OpenAI’s 2024 GPT-4o Launch: Voice, Vision and Desktop Demos

At OpenAI’s May 2024 spring update, Mira Murati introduces GPT-4o, a new flagship model that the company says brings GPT-4-level capability to free users as well as paid users. This video is a supercut of the presentation and its live demonstrations. The capabilities, rollout plans and prices discussed are from that launch, not necessarily current product terms.

Murati says the company will roll out the demonstrated capabilities iteratively over the coming weeks. She also announces a ChatGPT desktop app, presenting it as a way to use the assistant within an existing computer workflow.

Murati describes GPT-4o as faster than GPT-4 and improved across text, vision and audio. The previous voice experience combined separate transcription, intelligence and text-to-speech components, she says; GPT-4o handles voice, text and vision natively, a design intended to reduce latency.

The model is also being offered through the API. At launch, OpenAI says GPT-4o is twice as fast, 50% cheaper and has five times the rate limits compared with GPT-4 Turbo. These are the company’s historical launch comparisons, not current prices or limits.

Murati brings research leads Mark Chen and Barret Zoph on stage. Chen tells ChatGPT he is nervous about the live demo and asks it to help him calm down. The assistant suggests deep breathing, then reacts to his exaggerated breaths and coaches him to slow down.

Chen explains that, in the demonstrated voice experience, a user can interrupt the model rather than wait for it to finish. He also highlights quicker responses and the system’s ability to respond to vocal cues. The interaction is a live demonstration, not an independent test of emotion recognition or a guarantee about every conversation.

To demonstrate vocal expression, Chen asks for a bedtime story about robots and love for Zoph. The assistant starts a story about a robot named Byte, then changes its delivery as Chen asks for more drama, a robotic voice and finally a sung ending. The exchange shows requested changes in style during a conversation.

For the vision demonstration, Zoph asks ChatGPT to help with a handwritten linear equation using hints rather than giving away the answer. The assistant first says it can see the problem before he has shown it; he corrects it, and it waits for the image. That premature response is a visible mistake in the demonstration.

Once shown the page, the assistant reads 3x + 1 = 4 and asks what to do with the extra one. Zoph subtracts one from both sides, producing 3x = 3. When he needs another hint, it asks which operation undoes multiplication; he chooses division and arrives at x = 1.

The team next demonstrates coding assistance in the desktop app. Zoph explains that the voice interface cannot automatically see the computer screen in this setup: he selects and shares the code before asking questions. ChatGPT describes code that fetches daily weather data, smooths temperature series and annotates an event on a plot.

Zoph asks what the function foo does and how the graph would differ with or without it. The assistant says it applies rolling means to the average, minimum and maximum temperatures, reducing fluctuations in the plotted lines. Zoph then generates the plot and shares it for another question.

ChatGPT describes the graph as showing smoothed 2018 temperatures and a late-September rainfall annotation. It identifies July and August as the hottest months, with maximum temperatures around 25–30°C, and says the axis is in Celsius. These answers illustrate the particular shared plot rather than a general guarantee of chart accuracy.

For a request submitted by the live audience, Chen asks the assistant to translate between English and Italian in real time. He instructs it to translate each language into the other; the edited clip presents a brief exchange before the team says it appears to work. The available captions do not preserve a full translation transcript.

Zoph then asks the assistant to look at a selfie and say what emotion he appears to be showing. It describes a smile and suggests he looks happy and excited. This is an inference from an image in one demo, not a reliable measurement of a person’s internal emotional state.

Murati closes by saying the team wants to make the technology accessible for people to try, with the demonstrated capabilities planned to roll out over the following weeks. The supercut ends with the live audience applauding.