The GPT moment for video hasn’t arrived, but AGI may already be here

Less than a day after Google I/O 2026 closed, Oriol Vinyals, VP of Research at Google DeepMind and co-lead of Gemini alongside Noam Shazeer and Jeff Dean, joined Jacob Effron on Redpoint’s Unsupervised Learning podcast. Effron opened with a telling detail: hours before they sat down, a Google model had just solved a hard math problem. The conversation that followed was aimed at founders and investors, walking through the research behind the I/O announcements rather than the product demos themselves.

Vinyals’s résumé is part of the point. He was an early member of Google Brain, moved to DeepMind in 2016, and has been involved in many of the last decade’s deep-learning breakthroughs. The questions Effron queued up were the ones people building companies actually ask: whether world models are a bet worth making, how memory gets solved, where reinforcement learning goes next, and which layer startups should occupy. What follows is the argument, not the keynote.

Google shipped Omni, a world model, at I/O. Demis Hassabis has repeatedly described world models as a path to AGI, while other labs have put more energy into code and recursive self-improvement. Vinyals puts those two bets on different layers. A model that writes code to improve itself is one thing; the object being improved — a language model, or a multimodal model that understands the physical world — is another. He is actively working on the self-improvement side, but Gemini has modeled vision, language, and video together since before the program even had that name.

Why does Google care so much about world models? Vinyals offers a clean frame: language models have already distilled the knowledge humans wrote down into their weights. That path won because people put it on the internet, and user usage created a flywheel. Videos and images contain enormous knowledge too, and they are already in the training mixture, with some transfer and generalization. But the qualitative leap — the GPT moment for video and images — has not happened. Training on every video ever produced, without relying on text, and extracting the same depth of understanding language models get from words remains, in his view, one of machine learning’s core quests.

The hard part is linking concepts to pixels without an explicit language bridge. Gravity is the usual example: if you only watch video, can you derive the rules? The moment you add language, gravity is already in the weights from web explanations, so a correct answer is not evidence the model learned physics from images. Building labeled datasets that connect media to descriptions shrinks the available data, because nobody has transcribed every piece of media. Discrete representations and representation learning exist as early research, but they are not something Google can scale yet. If the direction unlocked, Vinyals says, the impact would be massive.

What makes Omni a world model rather than another video generator? A classical world model is representation learning: compress video into a set of concepts. What Omni shows is further along. Users can rewrite a video’s behavior in language — say “move forward” — and the model precisely simulates the action. The world model becomes a renderer of the world that you steer with words. Beyond generation, that object can be used for prediction before acting, with obvious applications in self-driving and robotics.

The link between world models and robotics runs both ways. Robot-collected data, expensive as it is, can flow into the model and strengthen the world model itself. In the other direction, the world model can generate training scenarios so robots do not have to spend time and money failing in the physical world. Vinyals is careful not to oversell the second path. Sim-to-real transfer is still an open problem, and the details humans take for granted when grasping an object — how hard to squeeze, how it feels, how it will move — demand extreme precision. Touch as a modality barely has data at all.

His forecast is therefore split. Fine motor control still has a clear gap. At the level of planning and coarse actions, though, these models will start accelerating robotics research. Hardware investment is already picking up, and the inflection comes first in places where the work becomes worth doing. The models do not need to be perfect simulators of fingertip physics to be useful at a gross level of behavior.

How would you even evaluate whether a model learned physics from video? Vinyals thinks out loud on the podcast. As soon as you add language, gravity knowledge is in the weights, so asking questions is not a clean test. One path borrows from unsupervised machine translation: align a speaking language model with a world model’s conceptual representations, then decode those concepts into explanations or even equations. He recalls alignment papers from around 2014, including work by Stefan Gauss and colleagues. A more indirect eval would ask whether you can induce movement in a complex system from its representation. Evaluations, he notes, are always the most important thing.

Agents were the other I/O headline. Google shipped Spark, a consumer agent widely read as an upgrade of 2024’s Project Mariner computer-use work. Vinyals attributes the capability jump to sequencing: get the model to a certain level, build a system around it, then jointly optimize model and system, and repeat. Spark looks narrow because it has rich context such as a user’s calendar and can help organize a day. Vinyals leans on a machine-learning regularity instead: components start general and specialize in applications. The Transformer was a machine-translation network; now it does everything from Omni to controlling a computer. Joint training across everything, he argues, beats narrow single-domain training. Specialization happens through the model’s intelligence and through user instructions, not by hard-coding a separate system for each task.

That logic makes him skeptical of the elaborate scaffolding now in fashion. Multi-agent setups, sub-agent delegation, and long-running loops are themselves a piece of code. In the limit, the model can write that scaffold on the fly, more token-efficient and higher quality than a fixed human-designed system. Reasoning models already previewed the pattern: whether to reason, and for how long, became something the model had to learn. The right to write the scaffold will eventually belong to the model as well. Reliability of long-running agents, in his view, comes from improving both the scaffold and the weights. Gemini 1.5’s long-context breakthrough is the precedent: working memory for ultra-long tasks has to be absorbed into the weights, because prompt luck does not hold forever.

Memory is where he sees a paradigm shift comparable to reasoning a year and a half earlier. Drawing on Demis Hassabis’s PhD work on memory systems in the brain, Vinyals splits memory into working memory and episodic memory, analogous to L1 and L2 cache. Transformers already handle working memory well — millions of tokens, theorem proving, gold-medal math. The open problem is consolidating knowledge across sessions. The current mechanism is to write an agent’s thoughts into files, organize them in directories, and retrieve with a basic search. The industry now calls this skills, but the mechanism is more general: the agent already has a computer, and the computer is its memory system.

He is explicit that this file-system, non-parametric memory is preferable to writing knowledge back into the weights. Google serves one model at scale; a different set of weights per user would be painful to operate. The picture he expects is two knowledge bases: a personal one per user, plus a shared layer common to all users of a given model, both enhancing the model without touching the weights. If per-user weights ever proved best, hardware investment might eventually make them feasible — but the practical path is non-parametric. He calls it a form of continual learning, and a shift on the order of reasoning.

Reinforcement learning has taken off in coding and math. Asked where the next explosion will be, Vinyals is first humble: today’s models are already so good at so many things that it is hard to point at a digital task and say it does not work at all. Clever prompting plus the right system can accomplish a great deal of what he calls digital AGI. When he called post-training greenfield, he meant the mechanics, not a capability gap. The compute poured into pretraining is still out of proportion to what post-training currently uses.

Go makes the underlying logic clear. In the AlphaGo era, a few moves into a game put the system in a configuration it had never seen. Environmental complexity made training data infinite and free; more hours of play meant more knowledge. Language models are data-limited. The missing ingredient is a source of infinite complexity that generates new situations for free. Whoever cracks that recipe, he says, opens a large direction — and it would be more satisfying, given how RL worked in games, to see it work in LLMs.

He cares less about verticals than about meta-capabilities: learning efficiently from experience, and instruction following, which he calls the ultimate capability — if you ask a model to build AGI, it either follows the instruction or it does not. His private eval, used for more than a decade, is to hand a model the Civilization instruction manual and see whether it can play. The test is dual: understand the rules, then get better while playing. The game has to be invented or otherwise out of training. Models are not that good at it, especially on games he makes up on the spot. Effron realizes he should not press for details, or the next generation of models will have the podcast in the weights. Vinyals’s joke: by then world models should be working, and the episode will be transcribed anyway.

Generalization is already happening. After moving back to the United States, Vinyals asked models about relocation and taxes and found the reasoning quality hard to believe as specialized training. The team still hunts for hard problems that induce deep reasoning and deep agentic behavior, but he is unsatisfied with verifiable tasks alone: most of what he wants models to do, he could not write a verifier for even with unlimited time. The way out is asymmetry. Creating a solution is harder than evaluating one. Math works that way; so does judging whether code produced a beautiful, engaging game. That gives him hope that models can serve as judges even without a strict verifier. In the quickfire round he admits the surprise of the past year: narrow, high-difficulty training on math and coding generalized far more than he predicted.

Effron asks the founder question: build at the model layer, or purely on top? Some companies now run their own RL above a frontier model; Cursor has gone as far as training its own base model. Vinyals answers in three layers. First, evaluations and data are the most valuable assets founders hold, and they are tightly coupled. Even without training a model, designing how to measure progress in your direction is hugely valuable — valuable enough that a lab like Google might adopt or monitor the eval. Post-training data is scarce; unique data will only get more expensive.

Second, start with the product. Pick a direction you actually believe in, especially if the large labs are not focused on it, then accumulate users, scale, and an understanding of the evals. Many companies follow that path: application first, then a decision about whether to use the data for post-training. Third, watch the treadmill. General models turn over every two or three months. A small post-training lead gets washed out by generalization, and you have to redo the work. A more durable angle, he says, is to build a complex knowledge base for the application and let the model use it through continual learning. That is cheaper than training weights, the uniqueness can protect you, and the capability itself will only get better — more scalable for early players.

Hassabis said at I/O that humanity is at the foothills of the singularity and that AGI could arrive in the next few years. Vinyals agrees and goes further. Seven years ago, before the LLM era, if he had been handed today’s models to experiment with, he would probably have declared them AGI. The definition keeps moving; being closer has made people more ambitious. By the expectations of only a few years ago, AGI is here in some sense. What he still wants, in his own mind, is the ability for models to truly learn from experience.

The named weakness is innovation. His team is experimenting with asking models for new machine-learning ideas, then implementing and deploying them; the coding step is already excellent. Innovating with taste is rare even among humans, and often a matter of luck: ten thousand people try, and history glorifies the one who was right. Move 37 was the canonical example in the previous world. He has not seen truly outstanding new ideas from today’s models; the combinatorial geometry problem solved just before recording is the closest recent case. Capabilities that are hard to evaluate, he notes, are also hard to climb.

Google’s position is hard to copy. Gemini already powers products across the company, and other groups have fully bought into the LLM era, so the modeling team can hand off a model and let others take it where it needs to go. Hardware procurement and end-to-end revenue give stability that lets Vinyals take a bit more research risk. He describes the posture as unfocused, but scalable because of how Google is organized. Gemini became a focusing, unifying force, helped by years of friendship with Jeff Dean, including trips they took just for fun. Research questions that will not make the next three-month training run still cannot drift too far from the frontier: Flash already beats a Pro from a few months ago, so multi-year calm is gone and the cadence is measured in months.

On compute, Twitter has asked why Google sells some of it to Anthropic if it is so bullish on its own models. Vinyals treats it as a multi-pronged equation: serving, training small models, training frontier models, and turning sold compute into revenue that gets reinvested. Hardware is a strategic asset; using all of it internally is not obviously the right trade. He recalls a room in 2013 or 2014 with Geoffrey Hinton, Jeff Dean, and Ilya Sutskever, deciding how many GPUs Google’s servers should have. Judgments about research were written into data centers years later. Being under the same roof, seeing the same things, was an advantage in the scrappy early days of deep learning and still is, with less uncertainty. Self-improvement has physical ceilings — energy, hardware, iteration speed. A remark from more than a year ago stuck with him: when a model writes English better than you, that may already be too good, and some ceilings are not worth reaching. His closing advice is plain. Whether you are a user or a builder, use the models, even for work that seems to have no connection to AI. They are already amazing, and they will only get better.