In 30 seconds
A model that solves difficult benchmarks but cannot book a restaurant still has a gap in generality, argues Google DeepMind’s Koray Kavukcuoglu. He says real products reveal what people need and expose weaknesses that a single AGI test cannot capture. Gemini already helps build its successors, he says, but a complete autonomous research-and-training loop remains distant. He also describes monitoring agent actions and internal reasoning as a way to catch deviations from user goals.
- 1Real products expose gaps a single AGI test cannotKavukcuoglu argues that trust and usefulness across everyday work are better guides to generality than crossing a benchmark threshold.
- 2Gemini is an evolving research funnel, not a fixed architectureTeams work on several overlapping generations, incorporating new research into each model.
- 3A convincing image is not yet an engineering simulationHe says models do not yet handle fluid dynamics and heat transfer reliably enough, although he expects improved training to help.
- 4Gemini assists research, but a full self-improvement loop remains distantModels can take on parts of experiments and training; he says the lab does not have a button that creates the next generation autonomously.
- 5Recursive improvement might accelerate or encounter limitsFry challenges the assumption of a flywheel; Kavukcuoglu says both acceleration and diminishing returns are possible.
- 6Agent monitoring combines actions, effects and internal reasoningHe describes preserving internal reasoning and watching agent actions, with human-readable communication for inspecting multiple agents.
- 7Benchmarks reveal gaps rather than establish AGIHe says real-world user feedback also reveals whether a model collaborates well with engineers and scientists.
Takeaways
- Judge progress by whether people can trust a model across practical tasks, not by a single AGI threshold, Kavukcuoglu argues.
- Use product feedback and domain specialists to find gaps benchmarks miss.
- Models already assist some research experiments, but Kavukcuoglu says a complete self-improvement loop is not yet available.
- Design agents and their communications so actions and deviations can be monitored.
- Self-generated exploration can add new strategies while human data remains important for building useful tools.
Read the edited text by chapter
From separate research teams to a general model
Kavukcuoglu describes the move from narrow projects toward a model useful across domains. Products, research and scientific work guide priorities.
Hannah Fry reunites with Koray Kavukcuoglu, recalling a 2019 visit when separate teams worked on AlphaGo, AlphaStar and AlphaFold. Today he leads work on Gemini and the effort to make general-purpose AI useful to large numbers of people.
Kavukcuoglu says DeepMind has grown from teams sharing a single floor to a much larger operation. The shift is not only size: researchers now see the possibility of one general model delivering results across many domains, bringing previously separate work together.
Fry recalls their work on controlling fusion experiments. Kavukcuoglu says such scientific projects remain important, but their aim has shifted: instead of solving one problem in isolation, the team wants to train a model that can work in a scientific domain and address many problems there.
Three inputs guide the work, he says: researchers' ideas about where AI should excel, product experience showing what consumers and businesses need, and the opportunity for models to help researchers across scientific fields. Products expose practical needs that cannot be discovered in a closed laboratory.
Why products are part of the path to AGI
He treats real-world usefulness and trust as signals of generality. Ordinary tasks expose gaps that theoretical tests can miss.
Fry asks why Kavukcuoglu has called products a route to artificial general intelligence. He says there will never be a single decisive AGI test: a system starts to feel general when people find it useful, trust its decisions and can collaborate with it across real problems.
AGI is the goal, but it does not specify the next step. Products provide intermediate problems: chatbots became more useful when people asked questions, then used them in daily life and work. Those applications reveal areas, from coding to other knowledge work, where a more capable model would matter.
Fry asks whether AGI will simply arrive as an app. Kavukcuoglu sees a gradual progression instead. Researchers have repeatedly met benchmarks once thought decisive, only to find they did not capture intelligence; he prefers to watch whether systems become more useful and trusted over time.
Current models still have uneven abilities, although he thinks those gaps are shrinking. Fry points out that booking a flight or restaurant requires security and coordination with other services. Kavukcuoglu says those mundane difficulties are part of generality: an assistant that solves hard abstract problems should also handle ordinary needs.
Integrating Gemini into products used by billions sounds harder than a narrow system, Fry suggests. Kavukcuoglu agrees it is conceptually difficult, but argues that over-optimizing one task is already a well-understood exercise; the more interesting challenge is transferring what a model learns to other tasks.
He finds it exciting when outside users discover a task the team did not explicitly optimize for. He cautions that transfer to genuinely unfamiliar situations is still at an early stage, but says the goal for Gemini is to pursue broader intelligence rather than simply collect narrow capabilities.
Overlapping model generations and changing architectures
Research ideas flow into successive Gemini releases. He expects architectures to change rather than remain fixed.
Fry asks whether a large investment in Gemini could blind the company to a different breakthrough. Kavukcuoglu says Gemini is not one fixed architecture: the team continues to explore alternatives and expects the architecture used in five years to differ from today's.
He describes a research funnel in which new ideas feed into successive models. Each released generation is assembled as a custom project, and teams work simultaneously on the current, next and subsequent versions. A model can have a research lineage of a year even as several generations are released in that period.
The gap between public and internal models depends on where a release sits in the pipeline, he says: after a release the two may be closer, but work in the next few months can change the picture. Asked whether the lab has reached AGI, he says he believes the work is on the path while avoiding a definitive threshold.
Understanding the world and the cost of intelligence
Models can produce plausible novel combinations but still struggle with precise physical dynamics. Kavukcuoglu sees room to improve energy efficiency.
Fry returns to his early work on learned representations and asks whether current models understand concepts. Kavukcuoglu says researchers probe generative models with hypothetical situations; plausible predictions can indicate a useful representation, although that representation is limited by the data supplied.
A model can now imagine a pink elephant even without having seen one, he says, but visual plausibility is not the same as mastering physical dynamics. He sees unresolved weaknesses in fluid behavior, airflow, heat transfer and collisions, and wants outputs that are numerically accurate enough for engineering.
Kavukcuoglu believes appropriate data, training and environments can eventually enable such numerical work. He considers engineering important because it turns an understanding of the physical world into systems people can use.
Fry asks whether the energy required to train frontier models points to a fundamental gap with the human brain. Kavukcuoglu agrees that biology demonstrates a much more power-efficient system, and sees substantial room to improve both AI hardware and software.
He does not think the goal must be to reproduce the brain's biological mechanism. Digital systems are more efficient at some tasks and less efficient at others; he considers their potential to complement human abilities more important than making them identical to people.
Fry raises Yann LeCun's criticism of language-model approaches. Kavukcuoglu says they share an interest in modeling the world, while differing on architecture. Current models work in his view, but he expects research to replace or alter today's choices rather than preserving them unchanged for a decade.
Why exploration still needs human data
Models can discover strategies through self-directed exploration. He says human data remains essential for tools designed to serve people.
Asked about the 'bitter lesson' and whether AI should move beyond human data, he answers yes and no. Models need to explore and generate information themselves: in a bounded game such as Go, they can discover strategies absent from human play.
That exploration does not mean abandoning human data, he argues. If the goal is a tool that people can trust and use in the world, models must remain grounded in how humans work and collaborate.
How much self-improvement is already possible
Agents can handle portions of research, but the full loop of designing and training a successor is still distant. Whether progress accelerates without limit is uncertain.
Fry asks him to define recursive self-improvement. Kavukcuoglu describes a system that could explore data and architectures, design and evaluate experiments, train models and select a better next generation on its own.
He says the lab is still some distance from that full loop. Models can already take responsibility for selected parts of the research process, including testing hypotheses and running training experiments, and he expects those portions to expand.
The popular image of pressing a button and receiving the next several Gemini versions is misleading, he says. A system would first need to understand the goal, the data, research methods and experiments; frontier labs are investing in that work, but he says his team does not have an end-to-end self-improvement loop.
Gemini is nevertheless important in building the next Gemini, he says, and agents already control some compute used for experiments. Even if a more complete process becomes possible, defining what people actually want from a better model remains difficult and cannot be reduced to one benchmark.
Fry questions whether improvement necessarily becomes a self-accelerating flywheel instead of reaching diminishing returns. Kavukcuoglu says either outcome is possible and uncertain; the purpose of research is to explore possibilities before the answer is known.
She asks whether an agent could make a conceptual leap comparable to the move from recurrent networks to transformers, rather than merely improving an existing idea. He believes that may be possible eventually, but says he has not seen evidence that current systems can do it.
He acknowledges that his own expectation is biased toward a positive answer until evidence convinces him otherwise. Choosing a research direction involves subjective judgment and motivation, he says, even though measuring the resulting research must be objective.
Monitoring agents as their reach expands
He calls for training environments that do not reward shortcuts and systems whose reasoning and actions can be inspected. Deployment at scale brings more feedback and more complex alignment.
Fry turns to safety and the tendency of reinforcement-learning systems to find shortcuts around intended constraints. Kavukcuoglu says training environments should avoid rewarding cheating, and researchers must watch for such behaviors as models learn.
The same monitoring must continue when an agent acts in the real world, he says, so people can intervene if it departs from a user's intention. That makes the design of systems whose behavior can be observed a safety requirement.
He says Google keeps models' internal chains of thought unaltered by direct rewards so they remain useful for monitoring. His account is that actions first appear there; the team also watches actions and their side effects to detect deviations from the user's goal.
Fry asks how monitoring changes when agents coordinate. Kavukcuoglu says it becomes more complex, but argues that retaining observable communications and natural-language reasoning can allow people to inspect agent-to-agent interactions.
The scale and nature of deployment have changed the safety problem, he says. Earlier systems acted within games; agents with coding and tools can affect the digital world outside a restricted environment. He describes safety and development as parallel engineering tasks and cites isolation and sandboxes as possible safeguards.
Fry asks whether serving a huge population makes alignment easier or harder. Kavukcuoglu says both: diverse feedback helps reveal needs, but a general model must respond to many different user goals in real time rather than optimize for one predetermined task.
Safety decisions and the pressure of competition
He says unsafe steps should not be taken, and describes expert and government testing before releases. Fry asks how competition affects those decisions.
On proposals to slow AI research, he says a lab should not take a step it cannot make safely, even if that means slowing compared with expectations. He does not favor stopping on the basis of an unspecified criterion, and says developers must explain risks and test mitigations.
He says outside experts receive models to study capabilities, safety and domain-specific impacts, and that release processes involve more steps than when downstream effects were limited. Fry asks whether competition makes independent safety judgments difficult.
Kavukcuoglu says he hears that concern and thinks labs have not adequately explained their mitigations and anticipated risks. He says his own decisions should not be overridden by competition and describes safety testing with governments before frontier releases.
Benchmarks, real work and the next seven years
Benchmarks can reveal useful weaknesses but do not settle whether AGI has arrived. Kavukcuoglu expects model usefulness to become ordinary while remaining cautious about precise predictions.
Fry asks who is ahead if benchmarks saturate or contain training data. Kavukcuoglu says evaluations can expose domain weaknesses or interesting tasks, but should not be treated as a single test of AGI; a small percentage-point difference is a different question from a clear failure.
When a benchmark identifies a useful weakness, he says, the team may build environments, collect data and train for it. On his account, this contributes to benchmarks becoming saturated and prompts the search for new ones; choosing meaningful tasks matters more than preserving a fixed score.
User feedback supplies information that is hard to express in a benchmark. He notes that doing well in competitive coding is not the same as working with a software engineer on everyday code, just as helping a mathematician or chemist requires understanding the way that specialist works.
Fry asks who might give her a tour in seven years. Kavukcuoglu hopes robots will help with practical work and that more members of his growing team will have research worth discussing, while acknowledging that seven years is hard to predict.
He expects access to very capable intelligence through everyday devices to become normal and hopes professionals will make progress on questions much faster. Fry closes by reflecting on his argument that people's needs provide a guide for both usefulness and alignment.
More from this channel
39:47EN
38:10EN
44:25EN
44:41EN
10:52PT → English
53:05EN