They trained a real-time video model in 20 hours and skipped the game engine

Sonya Huang had been playing Oasis that morning and opens with a label: a fully playable AI game engine. Dean Leitersdorf, Decart's co-founder and CEO, grants the technical headline first. A few weeks earlier the company had shipped what he calls the first video model that actually runs in real time and answers the player. You move, jump, break blocks, place blocks. The picture looks a lot like Minecraft. There is no game engine underneath it. Then he turns the claim around. That, he says, is not interesting.

Forget Oasis 1, he tells them, and think about Oasis 3 as a magic mirror. You look into it and ask for a sword in your hand; the sword is there, and it tracks as you move. Make it bigger, make it blue. Turn the whole scene into Game of Thrones, put a crown on your head, change the crown if you dislike it, jump, and the world jumps with you. Nothing like that has existed on Earth, he says, because it connects two things screens have kept apart for decades: the picture in your mind and the picture in your eyes. A display used to be glass that played something made in advance. In the mirror, the picture grows on the spot.

Shaun Maguire, Huang's partner at Sequoia, had already introduced the guest. Leitersdorf grew up between Israel and the United States, and at 23 he was the youngest person to take a PhD from the Technion — until his younger brother did it at 21. Decart, Maguire says, is trying to let people interact with their own imagination and other people's, and it is doing that by going fully vertical, from CUDA kernels up through model design and training to the pixels a user sees. The mirror is the destination. Oasis is the first object they put in front of it.

Huang asks the product question that follows any good demo. Is this social, a game, a world model? What is Decart? Leitersdorf answers with a question of his own: what problem does ChatGPT solve? Homework, she says. Emails, summaries, talking to computers, Maguire adds. The punchline is that ChatGPT does not solve any given problem. It overcomes a fundamental limitation. Humans speak unstructured language; computers speak structured language; large language models tore down that wall. Once the wall is gone, a hundred products can be built on the gap.

The mirror, and generative interactive video more broadly, is the same move for vision. Computers will see the world the way we see it, and show it back in ways we can understand. Cross that barrier and you can put next-generation Snapchat or TikTok on top of it, or a simulator for fighter pilots, or a new way of touching hardware. Founding a company in 2024, Leitersdorf says, is unusually fun if you are in one of those rare windows — once a decade, maybe every fifteen years — when you can overcome a limitation instead of picking a pain point.

Ninety-nine percent of companies solve problems. Their pitch decks open with the problem, the TAM, and the solution; the first two pages usually never change, and rewriting the solution five hundred times is still called iteration. Change the problem itself and people call it a pivot. He asks whether the Mac is a consumer product or an enterprise product, hardware or software, and what problem it solves. Try to list them and you get everything from gaming to Excel. A real platform, at the start, often cannot say what it is.

Game engines exist so one person can build a world and another person can enter it. Unreal and Unity have done that work for years; film productions now use Unreal for effects. The gift is consistency and precision. The cost is that changing the world is slow. Players, Leitersdorf has noticed, cannot leave a platform alone. Someone shipped a Minecraft mod that puts Pokémon in the forest. The platform is there, so users want to put their hands on it.

If what is running the environment is an AI, the grammar of a mod changes. You can talk to it the way you already talk to models. Turn this Elsa-themed, and the scene flips. Add a flying elephant, and the elephant is not a sticker: you can punch it and it punches back. He calls the direction interactive modding, done live, with words. For this trend to replace engines it has to become programmable — a machine on which one person builds worlds and another plays — and that day, he says, is coming. Programming will be words. Even people who can write code will iterate much faster.

The summary is blunt. Modding gets orders of magnitude faster, and it stops being something you download from someone else. It becomes a sentence you say to the world while you are inside it.

Huang asks what actually goes into real-time video inference, because she has not seen another model do it. Leitersdorf rewinds to the summer, three or four months earlier. Headlines said that when NVIDIA's Blackwell chips arrived, real-time video would arrive with them; Hopper H100s simply could not. He heard it from several directions. For about two weeks, everyone was repeating the line. H100s can do it, he says. They did it. You have to do two things at once.

On the model side, not every video architecture can be interacted with live. You have to train differently, and the architecture has to look different — not a revolution, but changes you cannot skip. On the systems side the work is uglier. They write their own CUDA kernels. They threw out PyTorch's garbage collector and rewrote something like half of it from scratch. Do only the systems half and you still do not have a model that wants to be driven by a keyboard. Do only the modeling half and the machine will not run in real time. Wait for someone else to finish the other side and you wait.

The model itself is not a mystery. It is isomorphic to the Soras of the world; the prompt is user actions instead of a sentence. Keyboard state and past frames go in, the next frame comes out. Getting the data takes a step ordinary video models skip: raw gameplay has to be labeled, frame by frame, with the key that was pressed. The team played for a while, recorded it, trained a small model to do the labeling, and ran that over the rest of the corpus. He stresses that this does not need much data. A small model is enough.

Is Oasis a world model or a pile of pixels? Right now it is purely pixels, end to end, from keystroke to frame. The other path would plug into Unity or Unreal with text-to-3D meshes — more consistent, an object that looks the same two hours later. Pixel space is more flexible: the elephant's tail is too big, you say so, it changes live. He thinks the two directions converge, maybe in two or three years, as a pair of transformers: one that holds game state like an LLM, taking the current state and a new action and emitting state changes, and one that renders that state to pixels. He wants to build both. Maguire notes they are still some distance from that merge.

What caught Maguire's attention is the ambition to be completely vertically integrated. He means it literally: electrons in logic gates, representations below assembly, CUDA kernels, model design, training, and the pixels on the user's screen, every layer being optimized. A company that does that, he thinks, keeps something like a 10x advantage over anyone sitting only on the application layer. Huang puts the obvious counter. NVIDIA has ten thousand strong engineers on the metal. A startup should be building the user experience and the viral loop. Why go down to kernels?

Maguire answers with Google, a company he has been studying as a business-model obsession since he was young, and where he later worked. People misread the original moat, he says. PageRank was a beautiful insight and a basic graph-theory idea, published, and quickly copied. The real advantage was that Larry Page and Sergey Brin were among the best in the world at distributed systems and low-level optimization, and they saw early that other search engines were buying expensive Sun servers for fault tolerance. Google bought cheap commodity hardware that failed all the time. You needed about five times as many flops to get the same performance, but the cost per flop was something like a fiftieth, which is a 10x cost edge if you can live in the wreckage.

The frontend users saw was a white page and a box, worse than Yahoo, which at least had chat rooms and flash. The magic was the backend. Sequoia, he adds, led Google's Series A, NVIDIA's seed, and had a piece of Apple's Series A — then Huang kills the commercial. He is careful: Decart has not done jack yet, and it is a long way from earning the comparison. His operational point still stands. A delightful mirror is a simple surface. You cannot deliver it in the next year, he thinks, unless you go all the way to the bottom, and these are the only people he has seen doing that. If it is not real time, the product does not work.

Leitersdorf cannot brag the way Maguire can, so he tells a different Google story. He was three months old when the company was founded. Talking recently with a very senior Google executive about unreliable GPU clusters, he heard an analogy: training today is where CPUs were in the 1990s, before Kubernetes, before VMware, when servers just crashed. On any hyperscaler or GPU cloud, a model of Oasis's scale dies every few hours, often for reasons that make no sense. One node crashes because two other nodes have dust on the cable between them, and no error message points there. Reboot fails. You start pulling nodes until the run stands up. Pretty much everyone sees this except Google, which built down to the hardware, and OpenAI, which had years to grind on reliability.

Most companies do not want the dirty work, so they pay twice: a premium, and time. Broken hardware always shows up before stable hardware. NVIDIA, the clouds, the software — they will make training stable in a year or two. That is not the state today. Decart decided to eat it. Oasis 1, the model in the demo, converges from start to finish in 20 hours. Friends at top labs were shocked. Their diffusion runs for a similar model usually take around two weeks, both because the systems layer is not optimized and because the job crashes every few hours or days. Decart can hold a run end to end, or for a week or two, without coming down.

The internal document is now about 200 pages: everything that can go wrong when you train a model, from "if you see this error, tell the operators these two nodes are sick" to stranger causal chains. One of the simpler stories, he says, is this. During an Oasis run they noticed CPUs sitting at about 3 percent, so they started generating synthetic data on the same cluster. The GPU cloud was stunned; it felt like 200 percent utilization — GPUs, CPUs, even InfiniBand moving data. Then training died on a missing lock file in the data loader. Synthetic traffic had eaten network bandwidth; Python's loader swapped a lock file that usually lived on the network onto local disk; nodes disagreed; the loader crashed. He figures the team would correct his details. That is the day to day.

Maguire offers another "simple" one. They got a new cluster before storage was installed. The GPUs had only a sliver of onboard memory; most teams would have sat and waited. Decart glued the SSDs on each node into a mini fake distributed filesystem and started training anyway. Leitersdorf calls their setup the best place on Earth to train a video model, and then immediately qualifies it: a video run needs storage, networking, and everything around the GPUs, and the clouds are still far from that. They will stabilize over the next half year, he assumes. Until then, the people who can improvise keep the job on the rails.

Vertical integration, as Leitersdorf has learned it from people like Maguire, usually gives you two things: cost, and speed. Maybe a third, because things often come in threes, who knows. In this AI cycle both matter; speed matters more. The ugly problems will get solved, on someone else's calendar. He cites a piece in The Information: people who leave Google to start companies discover that nothing works outside, because everything worked inside. Clouds and middle-layer vendors will eventually hand you storage and training plumbing. If you do the stack yourself, you can reach the market a year before everyone else, maybe two.

Tech moats, he adds, do not last. Bing is a decent search engine; Microsoft is a giant and has been at it for years; Bing and Google are still not in the same league. Google and OpenAI's move is to grab a technical lead and convert it into some other moat as fast as they can. Sequoia could park the money in the bank, take interest, sit on a beach for two years, wait for the ecosystem to mature, and come back to build the same company. Someone else would already be sitting in the chair.

Asked for the moat, Leitersdorf wants both horizons. Short term it is technology: the best systems layer, the model layer, the whole stack in-house. Long term he has been watching a weaker, newer network effect — what people say on TikTok. Character.AI is the case he learned from. After it took off, competition arrived almost immediately; the technical lead lasted about half a year, until Meta released open-source models and other people could run something similar. Character.AI was still vertically integrated, still maybe 10x cheaper. What impressed him more was the For You page. Search a competitor and you get a video of them, then a hundred of Character.AI. Open the competitor's video and the comments are Character.AI. Ask a random user and they do not know anyone else exists.

It is like brand, and more in your face. Twenty years ago brand was whether friends or parents mentioned you. The younger generation lives on TikTok, so the reputation hits them directly. He is not sure whether to call it network or brand, or whether it can last the two or three years you need before a real long-term moat — Google-scale brand, or distribution — exists. Maguire's version is more physical. Hardware is always the best moat, because you cannot change the timescale of atoms. Google turned a software and distributed-systems edge into optical interconnects inside the data center; a paper trail under the name Jupiter Rising, switches most outsiders said could not be built, roughly doubled performance and cut power by something like 30 percent. Even with AGI and a billion Optimus robots, fabs and power plants still take longer than software.

When they were founding Decart they called the moment a golden ticket, once in a life, and counted three giant companies you could try to build: an NVIDIA competitor, the next AWS, or new experiences. The first two are brutally hard — NVIDIA is a supply-chain giant, and in the cloud the default winner if everyone else loses is still the big three plus Oracle. They started with experiences, on the bet that the shift is large enough for a trillion-dollar company in five years rather than thirty. The chip path is a strong second; one day, he says, they will go back to it. The endgame language comes from Sequoia's James: generative experiences, GX, "UX is dead, long live GX." Decart is a generated-experiences company, building the wave that is supposed to touch every person on the planet. The last titans took a long time to rule. This round, he thinks, will be faster, and will still take at least five years.

In the rapid-fire close, favorite AI apps are ChatGPT and Character.AI, because he wants entities that hold a relationship — friendship or utility — with hundreds of millions of people. He first programmed at thirteen, writing bots for RuneScape, lasted six years, then used a downloaded cheat and was banned 24 hours later. AI-generated games versus novels? The first thing you will get, he says, is a platform for humans to use their own creativity, because AI is still far from making creative content. Favorite scientist is Leonardo da Vinci, scientist and engineer who knew how to raise money from the VCs of his day, who were kings; Tesla and Descartes are close seconds, and Decart is named for Descartes because "I think, therefore I am" rhymes with what AI is trying to do. Maguire refuses a congratulations: they have not done jack yet, go build something insane. Leitersdorf's answer is short. There is no celebrating small wins. You do not celebrate until you really win.