ElevenLabs Was Supposed to Be Roadkill

Pat Grady opens with the prediction that haunted ElevenLabs’ breakout years: the company would be roadkill once foundation-model labs added voice to their multimodal stack. Staniszewski, co-founder and CEO, does not pretend the fight is over. The advice that actually held, he says, is the unfashionable one. Stay focused. In their case that meant audio as a company, as research, and as product, all the way down.

He gives the research credit to his co-founder Piotr, “a genius” who made some of the first real innovations in the space and then hired a rockstar team. When they started, almost nobody serious was working on audio. Researchers crowded into language models and images, where results showed up faster and felt more glamorous. Diffusion and transformers had not been applied to speech with much care. ElevenLabs brought those ideas over, and for the first time a text-to-speech model could read the context of a passage and deliver it with believable tone and emotion.

The model was only the starting point. Staniszewski keeps returning to a second layer that hyperscalers still underweight: how you deliver the experience. Audiobooks, voiceover, film translation, speech inside agents, and the whole conversational product are what let them keep winning against foundational models. A great checkpoint that nobody can actually use is not a business.

Staniszewski and Piotr met fifteen years earlier in an IB class in Warsaw, took every course together, and bonded over mathematics. They later lived together, studied together, worked together, and traveled together. “Fifteen years in we are still best friends.” Building the company battle-tested the relationship. The first intense sprint was supposed to last four weeks, then another four, until they realized the horizon was closer to ten years and there was no room for anything but ElevenLabs. They still make time to talk about life, not only work, because a co-founder who ignores the personal side cannot bring a best self to the office.

Before the company, Piotr was at Google and Staniszewski at Palantir. Their hobby was hack-weekend projects: a recommender that tightened around whatever you picked, a crypto risk analyzer that did not quite work, and, in early 2021, a tool that analyzed how you spoke and offered tips. That audio project was the first time they saw the whole map. Speech recognition and diarization existed. Speech generation still looked primitive.

The aha moment came late in 2021. Piotr was about to watch a film with his girlfriend, who did not speak English, so they switched on the Polish track. Every foreign movie they had grown up with was the same ritual: one male voice, almost no pitch, narrating men, women, and children alike. It is a horrible experience, and it still happens. They decided technology should let people hear content in the original, astonishing voices. The same gap existed far beyond Poland — most of the world’s media is trapped in English, and audio is the path across the language barrier. *Attention Is All You Need* made the timing feel real. A year into the company, the open-source Tortoise TTS repo showed that cloning a voice and generating near-human speech was possible, even if the model was unstable. That was the glimpse. Then they started from scratch, bringing transformers and diffusion into audio until the result could feel like a human voice.

What ports from text to audio is less than outsiders assume. Compute, data, and architecture all differ. Speech models are smaller, so a small lab is not automatically dead on FLOPs. The real gap is data. Text corpora more or less work if you ingest them. High-quality audio is scarce, often lacks an accurate transcript, and almost never comes with the third layer that current models need: *how* it was said — emotion, speaker, non-verbal sounds. That layer barely exists at high quality. ElevenLabs spent its early years building a speech-to-text model and a pipeline of human labelers to create it.

The model has to predict the next sound, not the next text token, and it has to look forward as well as back. “What a wonderful day” in a novel sounds cheerful until the next clause marks it as sarcasm, at which point the punch line, the stress, and the emotion all move. Voices are handled the same way. The team did not hard-code male or female or age. They let the model learn a voice encoding, then fused that style — calm or dynamic — with the textual context to produce the final audio. It is a different job from a text model, and it is why a vertical lab can still win.

Hiring looks different too. They went fully remote from day one because the planet holds maybe fifty to a hundred people they would admire in audio research, judged by papers, open source, and prior companies. Researchers sit extremely close to deployment so the loop from idea to users is short. A second caste of research engineers takes existing models, improves them, and ships them at scale — work other labs would simply call research. Around that sits a labeling org that traditional vendors do not offer: voice coaches train data labelers to mark emotion, then review the labels. The research-plus-research-engineer group is about fifteen people, small, high-ownership, and expected to carry a theme mostly alone.

Grady remembers the Harry Potter-as-Balenciaga video and other consumer explosions that traced back to an ElevenLabs voice. Staniszewski treats those accidents as strategy. Every new model goes out to the whole user base. Professional consumers adopt in hours and invent jobs the team would never have listed. Enterprises follow, demanding more product and more reliability, and then the next model starts the wave again. Book authors, drunken sing-alongs, and faceless YouTube channels were not on the roadmap. Users found them.

The first mini-viral moment was the public beta in late 2022 and early 2023. The product was a tweet-length text box. One author pasted an entire book, downloaded the audio, and uploaded it to a platform that banned AI content. Listeners thought it was human. Reviews were strong. He came back with a circle of other authors who needed the same thing. Around then they published a post about the first AI that could laugh, and early users piled in. Creators then invented “no-face” channels — narration over footage, no presenter on camera — and ElevenLabs supplied the voices as the format caught fire.

Late 2023 and early 2024 brought multilingual models and a dubbing product that finally matched the founding vision: take audio, put it in another language, and still sound like the same person. Someone tried to dub singing. The model had not been built for that. It failed in a specific way, like drunken karaoke, and that failure went viral too. In 2025 the agent wave arrived. With Epic Games they recreated Darth Vader’s voice inside Fortnite; millions of players queued to talk to him, including people trying to goad him off the rails. The product held, remaining both expressive and safe. Another full-circle moment: Lex Fridman’s interview with Narendra Modi, English and Hindi in the room, dubbed both ways so each audience could hear both men in one language. India watched both cuts. The United States watched the English one.

Grady sees Sequoia companies climbing from text agents toward Hunyuan-style avatars with ElevenLabs voices. Staniszewski’s belief has been the same since the first day: voice will be the fundamental interface for technology. It is the modality humans have used since the species began, and it carries emotion, intonation, and imperfection that let people respond to one another in ways text cannot. Clients kept bolting text-to-speech onto homemade conversation stacks, so ElevenLabs built a product that abstracts the whole loop.

The bet is already landing in a few industries. In healthcare, Hippocratic AI automates the calls nurses used to make — take your medicine, how do you feel — and structures the answers for doctors, reaching patients who only answer a phone. Customer support is moving fastest, from Deutsche Telekom to newer companies standing up voice inside the firm. Education is the one that lights Staniszewski up. An amateur chess player himself, he is working with Chess.com on narration that coaches you through a game, with a future in which Magnus Carlsen, Garry Kasparov, or Hikaru Nakamura talks you through the board. His prediction is simple: everyone will have a personal tutor in a voice they actually like.

On the consumer side, Time magazine’s Person of the Year package let readers read the piece, hear it, and talk to it. The company also built an agent with Richard Feynman’s family, aiming at a future in which you hear *Surely You’re Joking, Mr. Feynman!* in his voice and then ask him about the stories. Grady’s follow-up is the real bottleneck: the interface is not what fails. Knowledge and business logic are. Staniszewski agrees. Speech-to-text, an LLM, text-to-speech, and turn-taking are only the ticket in. Enterprises burn time on Twilio or SIP trunks, CRMs, and call-center platforms such as Genesys. Knowledge bases range from a clean source of truth to a mess that needs ElevenLabs engineers on site. Protocols like MCP help, and ElevenLabs now treats them as a default rather than a custom project.

Foundation-model vendors sit in the same stack as partners and, sometimes, rivals. ElevenLabs stays provider-agnostic on purpose. Customers already disagree about which LLM they want, and a cascade — if one model dies, fail over to the next — has worked extremely well in production. Neutrality also hedges a darker future in which a lab stops supplying the service, or data-sharing gets blurry. “Treat them as partners, happy to be partners with many of them, and if we are competing there’ll be a good competition too.”

Grady repeats last year’s meme: people who tout benchmarks are missing the point. Staniszewski says that is especially true in audio, where leaderboards and lived experience diverge. Customers care about three things. Quality comes first — expressiveness in English and other languages. Miss it and nothing else matters, though the bar is different for narration, agent dialogue, and dubbing. Latency is the life-or-death line for a conversational agent; the engineering problem is the quality-versus-speed trade. Reliability is what shows up at Fortnite scale: millions of concurrent players, and the system still holds.

How far is human-level, effectively zero-latency voice? Staniszewski’s target is aggressive. He wants to cross a voice Turing test this year, so a person would say the call felt like speaking to another human — and if not this year, early 2026. Sharp listeners will be harder to fool. For most people, he hopes this year is enough. The timeline depends on a fork in the stack. Cascaded models — speech-to-text, LLM, text-to-speech as three pieces — are what run in production today: reliable, capable of being very expressive, a bit slower and less contextually alive. True duplex, or “omni,” models train those pieces together. They should be quicker and more expressive, and less reliable. ElevenLabs is about to ship a duplex model. Latency can get good on both paths. Nobody, he says, has yet fused LLMs and audio well enough to pass the test, including OpenAI and Meta. The internal goal is to be first.

Five to ten years out, technology should recede so people can focus on learning and on one another, with voice as the handle instead of a screen. Education becomes a guiding voice for math notes or a native speaker for a new language. Travel becomes a Babel fish from *The Hitchhiker’s Guide to the Galaxy*: you keep your own voice, emotion, and intonation, and the other person understands. The carrier might be headphones first, then glasses, then some non-invasive neural interface — he does not think it will be a phone. Everyday life fills with agents that book restaurants, sit in meetings, and call support, which raises a new problem: how agents authenticate one another, and how you know the voice on the line is a real person.

Impersonation is the fear that always arrives. ElevenLabs treated provenance as a day-one feature: every generated clip traces to the account that made it, so the company can take action. Staniszewski thinks the next step is deeper than labeling AI. Devices will authenticate humans — this is Mati calling you. The company also moderates voice and text for fraud, and it trains detectors with partners including UC Berkeley so the work covers open-source and other commercial models, not only ElevenLabs. Open source guarantees a cat-and-mouse game. As they take a leading role in conversational AI, they spend more time on safety that helps good actors and starves bad ones.

The company is remote, with a London base. Europe’s advantage, against the stereotype of low drive, has been talent: small teams of people as hardworking as any Staniszewski has met, plus a chip on the shoulder that Europe should not stay behind. Being global from the first day — not a Polish, British, or American company — matches the product, which is audio in many languages, and it lets local speakers serve local clients. The disadvantage is the missing veteran layer. In the United States you are surrounded by people who have built, exited, and scaled, and you absorb which questions to ask just by standing there. Europe is still figuring out the ecosystem. Enthusiasm is back; the EU AI Act, in his view, will not help anyone accelerate.

In the lightning round he names Perplexity for digging into sources, Claude for deeper coding and prototypes, and Lovable for spinning a client demo. The AI person he admires most is Demis Hassabis: a researcher who still speaks in depth, a leader who can deploy at scale, the mind behind AlphaFold, and someone who stays intellectually honest and humble. His hot take is that cross-lingual communication is badly underhyped. The largest barrier between people is that they cannot understand each other. Text translation is only half of it; voice delivery is the rest. The device that makes that ambient does not exist yet. Headphones will come first. Glasses will exist, but not everyone will wear them. A non-invasive neural interface is the longer guess. Grady files the whole picture under ambient computing: technology in the background, humans in the foreground. Staniszewski agrees. The form factor is still the missing sketch.