NVIDIA’s Ming-Yu Liu on Cosmos, Physical AI, and Creating New Markets

Host Zhang Xiaojun greets listeners, introduces NVIDIA research vice president Ming-Yu Liu, and notes that they are speaking in Shanghai one month after Jensen Huang’s recent visit to China.

The interview opens with a description of Liu’s understated style and his unusual position at NVIDIA: a researcher who has become an engineering-minded leader. Jensen Huang calls him “GSD”—someone who gets things done—and Liu has just led the release of Cosmos 3.

Xiaojun frames the conversation around a two-decade research journey inside a large infrastructure company. Liu says model building requires a low ego, and that competition should not become a Squid Game.

Liu says the prosperity of research should be measured by how quickly ideas move between minds. A conversation can improve an idea, which another person can refine again; diverse perspectives make the work stronger.

Liu recalls that when the team finished Cosmos 1, they asked Jensen whether to continue. Huang told them to keep going until Cosmos 97—a deliberately long horizon that signals the company’s determination to build foundational technology.

Liu explains that Huang’s response was not to rush but to pace the work. Huang is disciplined about marathon projects, and the team reached consensus on the need for Cosmos before deciding specific ownership and execution.

The host greets Liu and asks whether he still speaks Chinese at NVIDIA. Liu says most team communication is in English because many colleagues do not speak Chinese, though small groups switch naturally when everyone shares the language.

Liu says he visits China two or three times a year because Physical AI is developing quickly there. He meets companies and researchers to exchange ideas and understand where NVIDIA can help.

On this visit, Liu has seen rapid progress in dexterous hands and other capabilities for Physical AI. He says every trip teaches him something about how quickly the field is moving.

Asked about his role, Liu says he is both NVIDIA’s VP of Research and VP of the Cosmos Lab. Around eighty to ninety researchers and engineers report directly to him.

The broader Cosmos effort includes two to three hundred people across dotted-line teams. Liu says he is responsible for the overall mission and direction, even when contributors report through other leaders.

Liu recalls that his job talk ten years earlier predicted deep learning would eventually dominate video generation over traditional graphics. At the time, computer-generated effects were the norm, but he believed data-driven systems could do things a hand-built graphics pipeline could not.

He imagined a system that could take a novel as input and create a movie, while automating controls that once required large teams. After joining NVIDIA, he pursued that vision through GAN research and learned to build models that could translate between images without paired examples.

Liu describes early unpaired image translation: models learned to map between domains such as horses and zebras without paired examples showing the same scene in both forms.

NVIDIA was not yet known as an AI company. Liu’s early projects helped establish applied AI research there: GauGAN let users paint rough semantic regions and turn them into realistic landscapes, helping the company show that it could build AI models as well as GPUs.

Liu says he and colleagues worked on StyleGAN and helped bring generative networks for media creation to prominence at NVIDIA.

Around 2019–2020, Liu became a research director and founded Deep Imagination Research with collaborators. The team continued pursuing his job-talk ambition of turning a novel into a movie.

During the pandemic, Liu’s team helped create a digital-human keynote demonstration for Jensen Huang. He says digital humans and related applications were advancing quickly, even though the novel-to-film goal still seemed distant.

Liu says the novel-to-film goal remained distant until compute and data became sufficient. While his team was building digital humans and other demonstrations, OpenAI released DALL·E, making him realize he had underestimated what was now possible.

He wrote to Jensen Huang arguing that NVIDIA could not fall behind in deep-learning image generation while customers still used GPUs for traditional graphics. He also urged Getty Images to partner, offering data in exchange for NVIDIA’s compute.

The partnership became Picasso, an AI foundry for media companies that had content and customers but lacked AI infrastructure and expertise. NVIDIA would help build a generative model and return it to the partner to serve its own customers.

Liu says this changed his work from producing models and papers to delivering something that could help customers succeed. He asked Huang for 1,000 A100 GPUs—an enormous request in the Ampere era—and the team had to solve the engineering challenges of connecting and training them efficiently.

As an early computer-vision researcher in generative AI, Liu worked with talented interns, including Yu Jiahui and Tim Brooks. Brooks already wanted to generate video, a direction that later became Sora, although the technology was not ready during his internship.

Seeing Brooks succeed at OpenAI made Liu happy, but also prompted self-reflection: he wondered whether he had failed to act boldly enough at the right moment. A group of researchers later wrote Huang to argue that NVIDIA needed to invest more.

That discussion led to Cosmos. Liu, Sanja Fidler, Bryan Catanzaro, and Jim Fan agreed that world models would matter to NVIDIA and to humanity. Huang asked Liu and Fidler to lead the project, and eventually chose the name Cosmos.

Liu had been running both Picasso and Cosmos, but realized he could not lead two major efforts at once. He wound down Picasso, helped partners find other support, and committed fully to Cosmos.

As Cosmos progressed from version 1 to 2 and then 3, more customers and internal teams began relying on it. Liu says the project reached a point where it could no longer be allowed to fail and needed significantly more resources.

At the beginning of the year Liu left NVIDIA Research and began reporting to senior software VP Dwight. Deep Imagination became the Cosmos Lab, reflecting the transition from exploratory research group to a central product and platform effort.

Liu says he felt happiness rather than jealousy when former colleagues made Sora. The achievement showed how much impact research could have, while also giving him reason to examine where his own judgment had been wrong.

Asked why OpenAI later shut down the Sora product line, Liu says it was likely a strategic choice under pressure. Coding agents can run for many hours and generate far more tokens than a short chat, while Anthropic’s coding products were producing strong economic returns.

He compares the choice to the Art of War principle that defending everywhere weakens every position. OpenAI may have decided that coding was more important at that moment, not that Sora was bad research.

Liu says NVIDIA is an infrastructure company, but it cannot design useful infrastructure by simply selling tokens or GPUs. To understand what application developers need, the company must walk the paths they may take and do some of the application work itself.

Chip and computer-architecture development takes years, so NVIDIA must research emerging applications well before the hardware can be changed. That research helps the company identify which problems matter and build better chips and infrastructure for AI customers.

Liu says he wants ambitious people with ideas to have the tools to realize them. NVIDIA builds and shares models so startups and researchers can develop applications, extend the models, or draw inspiration to create better ones of their own.

The company’s open models are not turnkey customer solutions. Putting them into production still requires substantial work, especially in robotics, where hardware and business challenges remain. Liu sees the models as a signal that NVIDIA is committed to helping the field.

By releasing successive generations, NVIDIA gives developers confidence to invest in their own specialized problems. Every company has limited resources; a reliable base model can let partners concentrate on the parts that differentiate their products.

Liu does not see this as competing with customers. Physical AI involves many difficult problems, and NVIDIA can help absorb part of the burden by building models that developers can use.

Asked why he did not pursue the hottest language models, Liu says his background is computer vision and media. NVIDIA already has strong language-model teams; he wanted to work on problems where his experience could contribute most.

Liu connects this to NVIDIA’s own focus on Physical AI rather than content creation. He saw that digital creative tools were advancing quickly, while society still needed physical systems for mobility, dangerous jobs, and household work.

Self-driving cars could preserve mobility for older adults, robots could take on hazardous labor, and household robots could handle chores. Liu says these real-world needs made Physical AI a more meaningful target for Cosmos than content generation.

Liu says Jensen Huang sets a strict bar for the Cosmos team. The first requirement is to help customers, even though NVIDIA usually supplies infrastructure rather than complete robotics solutions.

Because NVIDIA is not a solutions company, Liu says it can feel like trying to scratch an itch through a boot. The team has to visit customers and understand their real pain points before it can design useful models.

Huang also expects the work to have a large impact. He favors “zero-billion-dollar businesses”: ideas that may not earn money at first but could become influential markets if they succeed.

Liu says the team must choose the most important problems and aim to be the best at solving them. That ambition is difficult, but he sees Physical AI as large enough to justify the effort.

Liu grew up in Taipei and initially studied wireless communications at National Chiao Tung University. As a teenager he was curious about mobile phones, then began questioning whether incremental improvements to communication protocols were the kind of work he wanted to pursue.

As a junior, Liu looked at U.S. graduate programs and was drawn to research that could create a 0-to-1 change. He decided to leave Taiwan for the United States, surprising his parents.

Liu initially planned to study quantum communications. Before moving to the U.S., an internship at Intel Taiwan exposed him to image recognition and computer vision.

During his junior year, he looked at U.S. graduate programs and saw research that felt more like a 0-to-1 change. He surprised his parents by deciding to study in the United States, first intending to work on quantum communication.

An internship at Intel Taiwan introduced him to image recognition. Watching a computer infer what objects were present from pixels felt almost magical, and gave him his first real exposure to AI and computer vision.

At the University of Maryland, Liu found that computer vision was more interesting than quantum communication and joined Rama Chellappa’s lab. His first project identified people by their walking posture in video, and he gradually moved from recognition into segmentation, shape matching, and other vision problems.

Traditional computer vision required a broad technical foundation: optimization, support-vector machines, graphical models, energy minimization, and 3D calibration. Liu says the work often failed, which forced researchers to learn many different tools.

AlexNet appeared around the time Liu finished his PhD and changed the field. At Mitsubishi Electric Research Labs, his first U.S. job, he watched deep learning begin to replace the traditional methods he had spent years studying.

Rather than feeling lost, Liu was curious. In his final year as a PhD student, he organized junior colleagues to study deep learning even when some thought the area was too obscure to be worth pursuing.

His advisor had long treated generation as part of understanding, and Liu became interested in generative models. After seeing Ian Goodfellow’s GAN work at NeurIPS, he began investing heavily in the area.

A discriminative model can classify whether something is A or B, while a generative model can also reconstruct and describe the signal. Liu sees that ability to represent how something is made as evidence of deeper understanding.

As a PhD student Liu had also worked as an engineer at Intel, where he learned how much patience it takes to find a problem inside a large software stack. He says research starts by asking the right question, while engineering builds the system that demonstrates the answer.

Liu says research and engineering serve different purposes: research begins by finding the right question, while engineering builds a system that demonstrates an answer. He learned this distinction while working inside large software systems.

His path into AI was accidental rather than a childhood plan. Computer vision was easy to recognize as important even before he understood it deeply: the possibility of recreating human-like intelligence naturally raises questions about what people are and how they differ from other living things.

Liu recalls an early attraction to elegant mathematics. Computer vision relied on difficult optimization problems, where researchers could rarely prove they had found the true optimum and instead sought bounds on how far their answer might be from it.

To make proofs tractable, researchers often introduce assumptions that simplify the math but move the model away from reality. Liu says he initially favored mathematical elegance too much, then realized AI is an applied science and must stay close to the problem’s essence.

Liu identifies three lessons from his career. First, elegant theory must remain connected to real applications; second, a strong result needs an explanation that lets others see its value; third, large research efforts depend on communication and teamwork.

He jokes that some of his research was worth 100 points, but his explanation conveyed only 10. At NVIDIA he learned to show not just what he built but why it mattered and how it worked.

When teams grow, point-to-point communication scales badly: every person must communicate with many others. Liu learned to make goals and the project’s importance clear so contributors could understand the same problem and move in the same direction.

Liu and his collaborators built a public-facing painting demo from the SPADE method. Jensen thought the technical name was not accessible, so the team looked for a name that would make sense to users.

Liu’s GauGAN demo let people paint a few colored regions and turn them into a realistic landscape. The algorithm was called SPADE, but Jensen Huang thought that name was too technical and not user-facing.

The team planned to show GauGAN at GTC 2019, but it did not make the keynote. Media demonstrations and a public web app still brought the project to a broad audience.

The team renamed the demo GauGAN, blending the idea of a painter with the GAN era. Huang planned to announce it at GTC 2019, though the keynote ran out of room; media demonstrations still made it one of the event’s most discussed projects.

A public web app let people try the model themselves, and user feedback gave Liu a new sense of accomplishment. He realized research could reach far beyond other researchers who read papers and influence a much broader audience.

Liu traces his GAN work from CoupledGAN, which linked two generators and created an information bottleneck, to image translation between horses and zebras. The bottleneck encouraged the models to find shared structure across two distributions.

Generative systems can sample images, landscapes, and videos, but users also want control. Liu describes a progression from image inputs and sketches to segmentation maps and class labels, culminating in text-to-image and text-to-video.

Liu says the easiest control signals were an image, a sketch, or a segmentation map. Those structured inputs made it easier to guide generation before the team tried to control images with natural-language prompts.

Segmentation gives precise pixel-level structure, whereas text and images have very different forms. Text is more natural for people but harder for models to align with visual structure, so Liu built up from easier forms of control before attempting text.

Liu and colleagues predicted as early as 2016 or 2017 that language might be the first AI field to make a major breakthrough: text is a relatively pure symbolic signal, while images contain much more noise. Even so, he stayed with visual generation because that was where his interests lay.

At NVIDIA, the team moved from image translation to video translation. Their vid2vid work transformed simple, cartoon-like driving-simulator frames into realistic scenes, an early step toward the world-model work they do today.

Liu says early video-translation work was a step toward today’s world-model research. The team used simple driving-simulator images as input and generated realistic scenes, connecting visual generation with a simulated physical environment.

Liu says researchers should favor simple, scalable approaches—the “bitter lesson” in retrospect. GANs were difficult to train, so he and interns Jiahui Yu and Jeremy Bernstein tried to scale and stabilize them with large GPU clusters and new algorithmic ideas.

The experiments did not solve GAN scalability, but they helped shape later work. Bernstein continued along the path of optimizer research that led to Muon, while Yu explored ways to distribute discriminators across GPUs.

Liu and interns Jiahui Yu and Jeremy Bernstein tried to make GAN training stable at scale. Their experiments explored larger GPU clusters and ways to distribute discriminator work across devices.

Although the GAN experiments did not produce a scalable recipe, Liu says they informed later research. The team kept what it learned and moved toward approaches that could benefit more from scale.

After years of work, Liu concluded GANs were not scalable enough and moved the team to diffusion around 2021. The group then progressed from images to multi-view generation, 3D, video, and Transformer-based systems.

Liu says paradigm shifts can make previous work feel wasted, but each stage contributes to finding a simpler explanation of the world. Cosmos 3 likewise unified several earlier models into one, after years of investment in each direction.

Liu says research improves when ideas pass between people with different perspectives. In industry, interns help broaden the discussion beyond a team’s usual circle and bring new directions to senior researchers.

Working in industry instead of a university gives Liu a narrower circle unless he actively broadens it. NVIDIA’s interns bring different backgrounds and new ideas, helping both senior researchers and the students grow.

Liu joined NVIDIA in 2016 after spending several years at Mitsubishi Electric Research Labs. He had become convinced that deep learning depended on compute, while Mitsubishi emphasized algorithms and required him to argue repeatedly for GPU budgets.

When a mentor left Mitsubishi for Apple, Liu reconsidered his own path. NVIDIA stood out because it made the GPUs he kept asking to buy, even though his parents had barely heard of the company and preferred more famous technology firms.

He believed compute and data were the key resources for unsupervised learning, since his methods did not need labeled examples. His former manager later admitted that whenever Liu won an argument for GPUs, he also bought NVIDIA stock.

Some papers that appeared to involve Mitsubishi and NVIDIA were simply work begun at one company and accepted after revisions at the next. Liu says joining NVIDIA was a normal job search, not the result of a prior collaboration.

When he arrived, NVIDIA Research had fewer than a hundred people and focused mainly on graphics. At an offsite in Monterey he met researchers who later became central to StyleGAN, and learned from their extreme attention to experimental detail.

Liu recalls that NVIDIA Research was still small when he joined and many researchers worked on graphics. He met the people who later developed StyleGAN and learned from their careful experimental practice.

Liu says researchers earn access to company resources by understanding company priorities and explaining why their work will matter. Researchers often believe their own problems are the most important; a company must also consider its customers and shareholders.

For Jensen Huang, the company’s success supports the families of its employees. Liu learned to show leaders who may not understand AI how research can drive real change and help the business.

Liu says company research needs to account for customers and shareholders as well as scientific curiosity. Researchers must explain how their work connects to the company’s priorities.

Liu’s experience at Mitsubishi shaped this lesson. The lab once supported ambitious blue-sky research, but later leadership questioned its impact and many prominent researchers were let go or left for universities and other companies.

The research that survived was more closely tied to Mitsubishi’s business in cameras and robotics. Liu learned that blue-sky research and company goals can align, but researchers must make the connection visible.

When a company expands, it may tolerate more exploration; during a downturn it asks why an investment matters. Deep learning is expensive, so researchers need to explain the value of their work even more clearly.

Liu deliberately tried to turn research into products, but says AI alone cannot change everything. As model capability converges, companies need to integrate models with their products, platforms, and customer feedback.

Liu learned from Mitsubishi’s research lab that blue-sky work needs a visible link to the business. During downturns, leaders ask why a costly research investment matters and which problems deserve resources.

Google’s strength, he says, comes partly from the way Gemini connects to Search, Docs, and other services. Usage creates feedback, strengthens the product, and gives the company more reason to invest.

Early researchers are trained to prove their methods beat benchmarks, which can encourage high-ego competition. Liu’s focus has shifted toward what he can build to help the whole field improve.

He says many talented interns have been better than he is, and sees that as positive. His goal is to make useful contributions, not to claim he is superior to others.

Liu says people and ideas move between leading labs, so it is difficult for one company to stay far ahead indefinitely. As capabilities converge, established companies can integrate models into broader products.

Liu expects frontier labs to keep pushing intelligence forward, but doubts one company can remain permanently far ahead. Talent and ideas move between organizations, and competitors learn from each other.

If model capabilities converge, larger companies will compete through integration with their ecosystems. Startups can take on problems too small for giants, finding underserved needs that may grow into important markets.

A startup should not simply imitate OpenAI or Anthropic, whose funding, customers, and scale are hard to match. It needs a distinct problem and a conviction about where its own resources can make a difference.

Startups can find opportunities in problems that are too small for large companies to prioritize. Liu says a new company should choose a distinct need instead of trying to copy the largest frontier labs.

Liu advises founders to focus on areas where their own resources and conviction can matter. NVIDIA itself grew by pursuing a different opportunity rather than imitating a larger competitor.

Liu says his promotion to vice president came from helping NVIDIA understand AI’s importance and repeatedly delivering useful work. He believes trust grows when research contributes in practice.

Liu says he may be the first NVIDIA researcher to rise to VP. In his early years he was promoted quickly because he helped raise the company’s profile in AI, explained the technology’s implications, and consistently delivered work in practice.

Leading Cosmos means overseeing hundreds of people and millions of dollars in compute and storage costs. Liu believes the company entrusted him with the project because he earned that trust over time.

A researcher’s need for recognition can conflict with a manager’s job. Liu says he still reads papers and reviews code to avoid becoming disconnected from the core problems and to understand who is contributing what.

Staying technically grounded helps him support the team and judge its direction. He does not want to become the best implementer; he wants to retain enough context to see the hard problems clearly.

Liu names Rama Chellappa, Oncel Tuzel, and NVIDIA colleagues such as Tero Karras as important influences. From Karras he learned to pursue one idea deeply and extract its essence rather than chase a large number of papers.

Huang’s curiosity is one of the traits Liu admires. Huang studies unfamiliar fields, asks researchers to explain what they are learning, and is willing to make long-term investments when he sees a strategic opportunity.

Liu says Jensen Huang has had a tremendous influence on him. Huang regularly sent him research papers to study, and Liu was struck by Huang’s curiosity and willingness to enter fields he did not initially know, from graphics to deep learning.

Huang uses first principles to identify the essence of a problem, then makes long-term bets. Liu says watching him invest in deep learning and CUDA changed Liu’s own tendency to make shorter-term decisions.

Huang reads a remarkable number of employee emails, looking for signals that may matter to the company. Liu sees prioritization as a daily discipline: distinguish what is important, then execute without letting personal relationships distort the choice.

The volume of Liu’s work has grown from two or three research projects to constant shifts between subjects. He compares good decision-making to computer science’s principle of laziness: do not compute what you can avoid, and do not spend limited energy on irrelevant work.

Liu says research leadership has forced him to become selective. The computer-science idea of laziness—avoiding unnecessary computation—also applies to deciding which projects deserve his limited time.

Liu maintains his energy through sports and decades of Chinese martial arts practice. His teacher Xu Ji was part of a lineage tracing through Liu Yunqiao to Li Shuwen, the famed “Divine Spear.”

Liu says his martial-arts practice is about self-control, not fighting. He has trained for decades under Xu Ji and values the discipline of practicing forms and stances alone.

Liu trains alone in the morning, practicing stances and forms, and recently added ice plunges to his routine. He says martial arts taught him whole-body control and self-discipline, not how to display destructive power.

In the workplace, Liu can be direct with researchers who are falling short. He sees candid criticism as a way to help people improve, though he recognizes listeners may experience it as harshness.

Cosmos is Liu’s sole project, and he sees it as the group’s mission for the next decade. Huang’s “Cosmos 97” remark expressed a commitment to keep building the foundation for Physical AI over many generations.

Liu says world models can help embodied AI generalize beyond training data. Cosmos aims to contribute in three ways: provide better data, a better starting point, and a better environment.

The data a robot has seen covers only a small part of the world. A model could generate useful examples that were not collected in reality, helping developers train for situations that are rare or expensive to capture.

A world model that understands patterns in the physical world can predict what may happen next and provide a stronger starting representation for policy models and other systems.

People learn through interacting with their environment, but real-world experiments are costly and time is limited. A realistic simulated environment—the “Matrix for robots”—could let agents learn much faster.

Liu says the hardest part is building an environment realistic enough to produce useful signals. Cosmos began around March 2024, after Sora made the opportunity clearer, and Huang trusted Liu to lead the effort.

Liu describes a third kind of world model: a system that reconstructs a three-dimensional scene. This can let a robot or agent explore a virtual space and test plans before acting in reality.

He says no single definition of “world model” fits every use. Some systems predict the future, others explain why an event happened, and still others reconstruct the surrounding environment.

Unlike an LLM, which predicts the next word or token, a world model can describe many kinds of physical processes. Cosmos is intended as a foundation developers can adapt to their own problems.

The name Cosmos reflects a goal broader than content creation: model the physical world so developers can solve real problems. Liu says Physical AI became attractive because aging populations, labor shortages, and quality-of-life needs create durable demand.

Liu says Cosmos is meant to help developers who cannot afford to train a world model from scratch. NVIDIA wants to provide a starting point so different teams can build models suited to their own applications.

He describes NVIDIA as an enabler for teams working on different views of the world, including Fei-Fei Li’s World Labs and Yann LeCun’s AMI Labs. The company wants the wider ecosystem to succeed.

Physics itself can be understood as a set of models explaining how the world works. In deep learning, large-scale data can help systems interpret new events from video and other observations.

World models can also reconstruct 3D environments, enabling planning and virtual exploration. Liu previously worked on reconstruction but now focuses on understanding and generating the physical world.

Liu says a model trained on large amounts of video can learn patterns that help it understand new events. World models can therefore support both prediction and interpretation, depending on the developer’s goal.

Because “world model” covers many uses, Liu doubts the term can have one precise definition. He contrasts it with an LLM, whose function—predicting the next token in language—is more specific.

NVIDIA calls Cosmos a world foundation model because it provides a base for companies to build world models suited to their own needs. Many Physical AI developers cannot afford to train such a model from scratch.

Liu describes NVIDIA as an enabler for teams such as Fei-Fei Li’s World Labs and Yann LeCun’s AMI Labs. Different teams pursue different problems; NVIDIA wants to help them succeed with its tools and open models.

NVIDIA’s Cosmos team initially built separate models for prediction, visual transfer, reasoning, and action. Liu says those models were solving different parts of the same underlying problem: learning how the physical world works.

The team recognized that maintaining many specialized models created a burden for users. Liu says different models can be combined when they draw on the same representation of the world.

Huang encouraged the team to reduce the number of choices. Liu compares it to a store that sold more after removing excess options: a simpler product can make it easier for customers to decide and build.

When Cosmos began, the team first built several models for different needs: Predict for forecasting, Transfer for realism, Reason for explaining events, and Policy for action. Over time, Liu saw them as different views of the same physical world.

A foundation model learns from many streams of data. Liu reasoned that if the models were all learning the same world, combining their data could raise the overall capability ceiling.

At first, Liu estimated they might need 22 models to serve partners. Huang compared the idea to a luxury store that sold more after reducing the number of items on display: too many choices confuse customers.

Each released model also creates maintenance work. The team must help users deploy it, troubleshoot issues, and improve it; maintaining many separate models would dilute effort and slow iteration.

That led to Cosmos 3, a unified world model. The shift from Cosmos 2 to 3 was substantial, while version 2 had mainly improved data and added policy capabilities.

The team started executing on Cosmos 3 around October or November after finishing version 2.5, and spent roughly half a year on the release. Liu says model development decisions are made months in advance, so fast changes are difficult.

Some reviewers said Cosmos 3 had few entirely new technical ideas but validated many details at scale. Liu agrees that combining known components is not trivial: scaling a mixture transformer across audio, video, and action depends on making the details work.

Liu cares more about whether a method solves real problems than whether it is a revolutionary invention. The paper also makes the work transparent: NVIDIA open-sourced its model, training framework, and some synthetic data to help the Physical AI community.

Liu says NVIDIA released the model and training framework openly so the Physical AI community can inspect the work and build on it. He wants the research to help developers beyond NVIDIA’s own teams.

Liu says language, video, audio, and action describe different aspects of the same world. Vision is not the only signal; touch could matter too, although tactile sensors and dexterous hands are still developing.

Some models treat language as central, as in chat and coding; others center video, as in video generation. Physical agents differ because they act on the world, so Liu believes action should be a first-class signal alongside language and video.

Liu says language, video, audio, and action each reveal different information about the same world. Touch could also help, but tactile sensors and dexterous hands are still developing.

Language can express human knowledge and instructions, while video shows how objects and scenes change. Action connects the model to what a robot can do, and sound can reveal contact or other physical events.

Language carries human knowledge and lets people communicate with a robot. Video and audio reveal how the physical world changes, while action allows an agent to change that world in turn; Cosmos is built around this practical goal, not an abstract pursuit of AGI.

The team found that action prediction is stronger when it also predicts the resulting observation. Video, audio, and action can teach a model how the world changes after a robot acts.

The modalities can reinforce one another. Predicting the video state after an action can help train a policy, while a contact sound such as a click can clarify the moment an object touches a surface.

Detailed text descriptions can also make video generation easier by narrowing the number of possible outcomes. Cosmos 3 brings Reason, Predict, Transfer, and Policy together so the signals can complement one another.

Liu says there is no secret sauce: teams form hypotheses, run controlled experiments, study the results, and use what they learn to plan the next test. Progress comes from disciplined work and careful adjustment of data mixtures.

Cosmos 3 uses two towers: one for discrete information and one for continuous information. Reasoning grew out of a vision-language model, while generation and policy work use continuous representations, so separating them was a practical choice.

Customers often post-train Cosmos on their own video-action pairs or use only its reasoning capabilities with domain-specific data. Separate towers let them adapt one side without unintentionally changing the other.

A single connected architecture could make a customer’s generation data alter understanding performance, even if that data contains no understanding examples. Two towers are less elegant but currently offer a more usable system.

Liu says the long-term goal may be one tower, but the model must remain easy to use across robots with different cameras and hardware. Engineering simplicity matters only if customers can adopt the result.

Researchers are exploring unified architectures such as Transfusion that handle both discrete next-token prediction and continuous diffusion. Liu hopes for a simpler all-discrete or all-continuous design, but says experiments must determine whether either works.

Discrete systems can be easier to train; continuous representations have other advantages. In the end, deployment cost and how well customers can run the model matter more than theoretical elegance.

Liu invokes The Art of War: a model that tries to do everything may not excel at any one task. Strategy means deciding what to give up so a model can serve a particular application well.

Liu says discrete and continuous representations each have advantages, but the architecture must work for customers in deployment. A theoretical simplification is not useful if developers cannot run or adapt the model.

Liu wants Cosmos to help robots generalize. A useful system should learn a person’s preferred way to fold clothes after one or two demonstrations, then repeat the task in a real home.

Another test is whether a robot can read an equipment manual and then operate or inspect a machine in a factory. Liu says these examples capture the kind of generalization that would make Physical AI useful.

Future Cosmos improvements will target generalization and speed. That may require changes to architecture, training objectives, or carefully selected data, but Liu says the right choices must be validated experimentally.

He compares deep learning iteration to traditional Chinese medicine: keep detailed records, control variables, and learn from trial and error. Larger language representations may help, but Physical AI use cases require their own tests and trade-offs.

Liu says model improvements must be tested rather than guessed. The team keeps detailed experimental records, controls variables, and uses each result to decide what to try next.

He agrees that stronger language representations may help world models, as larger text encoders improved image generation. But Physical AI differs from chat and coding, so the value of more language capability still needs experimental validation.

Liu says model design should start from real customer pain points. When resources are limited, the question is which capability matters most for the task, rather than adding intelligence that the robot may never need.

For factory work or household chores, a robot does not need to solve mathematics or coding problems; it needs to perform its assigned job well. If a model is going to be commercialized, its operating cost becomes part of the design.

Liu expects models for physical work to differ from agent models for digital tasks. He compares the difference to tools for plowing fields and weaving: each tool should fit the job, rather than one architecture trying to serve every use.

Cosmos training already uses tens of thousands of GPUs, and Liu hopes the team can use more. NVIDIA organizes its models into digital agentic AI, represented by Nemotron, and Physical AI, represented by Cosmos; he sees both as important.

Cosmos also supports specialized work on autonomous vehicles and robots. NVIDIA’s vehicle and robotics teams build on the foundation with task-specific optimizations, and Liu considers those internal teams customers he wants to help.

Liu says Physical AI could become a huge market as populations age and manufacturers seek more autonomy. He avoids giving a company market-size forecast, but sees a broad range of applications that could create substantial demand.

Liu distinguishes three kinds of evaluation: fixed benchmarks, head-to-head comparisons, and tests with experts who have the actual pain point. For Physical AI, the third matters especially because each customer may face a different task.

Physical data is difficult to collect because each robot has a different body and setup. Liu identifies two broad categories: navigation from one place to another, and manipulation using hands or tools.

Navigation data transfers more easily across machines, while manipulation is more complex. Many teams are moving toward human-like hands because the human world and its tools were designed for people, and people themselves can collect useful demonstrations.

Cosmos 3 therefore added first-person data showing what human eyes see and how hands manipulate objects. NVIDIA does not intend to build every robot; it wants to provide transferable data and tools that help others build them.

NVIDIA’s advantage includes more than model weights: it also builds GPUs, the Thor robotics computer, and Isaac simulation tools. Liu says these resources help partners train and deploy their own systems.

Asked what he likes about Cosmos 3, Liu says the unified model outperformed the separate models it replaced. He is proud that a large team that once built several systems could come together and deliver one.

NVIDIA’s advantages include relatively inexpensive access to GPUs, long experience with Physical AI, the Thor chip, and the Isaac simulation stack. Liu says the company’s aim to help partners succeed also makes customers more willing to collaborate.

His regret is that the model had not fully converged before release. Time constraints left improvements unfinished, but the team plans to continue with Cosmos 3.1 and 3.2.

Huang’s “Cosmos 97” comment expressed a commitment to keep working on world models for decades. Liu’s ultimate image is a realistic Matrix-like environment where robots can learn by experiencing the world.

Liu says digital AI and Physical AI may need different architectures. Nemotron uses a hybrid design to keep long-running agent inference cheaper, while Cosmos explores Transformer-based approaches that include action.

The teams work closely, share authors, and may unify their models if that helps customers. Liu’s decision rule is simple: choose the design that best serves users rather than defending a particular architecture.

The Cosmos team includes nearly 300 contributors, creating an organizational design challenge. Liu wants goals aligned but decisions distributed, because one person cannot know enough or have enough time to make every technical choice.

If each subgroup optimizes only its own signal—action, video, or reasoning—the project can diverge into competing architectures. The team needs shared objectives and enough transparency for people to make decisions that help the whole model.

Liu says the paper’s final form was about 90 percent what he had envisioned. By describing a clear target to project leaders in advance, the team could converge while still allowing each contributor to play a different role.

Cosmos teams meet many times each week. Liu tries to make meetings engaging and participatory, but says there is no universal formula: too few meetings fragment work, while too many prevent execution.

He describes NVIDIA’s culture as a mix of top-down and bottom-up. Teams need to adapt their working style to the moment rather than take any one approach to an extreme.

A “ChatGPT moment” for Physical AI would mean the public can see an application that is plainly useful, not simply a model with impressive benchmark results. Liu says the timing is difficult to predict, though capital and entrepreneurial interest are growing.

Humanoid robots can reuse human demonstrations and operate tools designed for human hands, but adding legs and hands increases complexity. Liu expects humanoid forms to matter, while leaving the exact form factor to economic efficiency.

He does not expect a single sudden tipping point for world models themselves; they have existed for a long time and will steadily improve. Developers still need time to integrate new models into robots and real-world workflows.

Asked whether a Matrix for robots could make machines rebel, Liu says the metaphor is meant to illustrate accelerated learning. Human society builds and operates these technologies, and he believes shared norms and safeguards can keep development on track.

Liu adds that pain and suffering may drive people to improve, but he doubts silicon systems experience them in the same way. The goal is to improve human life while building guardrails that protect normal social development.

Liu says world models are tools for solving specific problems, not an end in themselves. Developers can use Cosmos to move faster; if an open-source partner builds something better, Liu sees that as a good outcome for the whole field.

He is not worried about making a copyable model. NVIDIA wants to help Physical AI arrive sooner, and a growing ecosystem can benefit many different companies.

Liu says open models are tools for developers, and NVIDIA is comfortable if a partner uses Cosmos to build something better. The company wants Physical AI to arrive sooner, with more teams contributing.

He does not expect a single model to become the next CUDA. Instead, he sees a larger stack combining Nemotron, Cosmos, serving software, and infrastructure, with different parts solving different needs.

Liu sees the next CUDA not as a single new model, but as the combination of Nemotron, Cosmos, serving software, and infrastructure. CUDA succeeded through a risky long-term investment that later found a major application in deep learning.

The path is not directly repeatable: the industry and its needs have changed. NVIDIA has to adapt and identify the most important contribution at each point, rather than carry a “hammer” and search for nails.

Over ten years Liu saw NVIDIA move from a company many people associated with graphics to one associated with AI. The shift was not smooth; the company’s stock and prospects had ups and downs, including anxiety when Google’s TPU appeared.

Liu says he joined for a simple reason: he needed GPUs for research and wanted to work where they were made. The company’s later success was impossible to predict, and he feels honored to have contributed along the way.

Liu recalls that he turned down an investment firm that offered four times the pay because he wanted to continue doing research. NVIDIA was less well known, but offered the compute he wanted.

Huang has changed physically and carries more pressure, but Liu says he still cares deeply about employees and reads their messages for signals. NVIDIA has grown enormously, moved from desktop GPUs to data centers, and added Physical AI to its priorities.

Huang often says he feels the company has only 30 days of cash left. Liu sees this as a discipline that pushes leaders to make the most careful decisions, not a literal financial forecast.

During a difficult period in the pandemic, NVIDIA cut travel and reconsidered duplicate software subscriptions rather than starting with layoffs. Liu says the company looked for unnecessary costs it could remove while keeping employees together.

Liu says he has never seen layoffs or forced bottom-performer cuts at NVIDIA. Employees can leave after performance reviews or by choice, but the company does not dismiss people simply because priorities changed.

He believes job security lets employees speak honestly and collaborate instead of fighting to stand out. The trade-off is that retraining people during a technology shift can take longer than hiring a new team.

NVIDIA improves efficiency by helping people work on tasks they find meaningful and are good at, when those tasks also matter to the company. Huang’s long-term culture gives the company room to prepare years ahead, as it did with CUDA.

Liu says NVIDIA plans years ahead rather than demanding every new technology win immediately. CUDA is an example of a long-term investment that later found a defining application.

Liu says NVIDIA avoids internal horse races. The company gives a team a mission and a high bar, trusts them to deliver, and then pushes them to improve through difficult feedback.

Building large models has brought Liu years of pressure: GPU resources and young researchers’ time are expensive, while results may fall short of what the team hoped. The fast pace since GPT’s rise makes the work demanding.

Liu’s routine has become more efficient: he wakes around four or five, exercises, and works through a schedule packed with meetings. He tries to protect the morning’s continuous time before the office day begins.

One NVIDIA practice is the “top five”: employees share the five most important things they are working on. This helps reveal signals across the company, including duplicate efforts that teams may not otherwise see.

Liu describes NVIDIA as an organism rather than a rigid chart. When a mission changes, relevant teams can align around it regardless of reporting lines; “mission is the boss” captures how the company reallocates attention.

Liu says mission alignment lets employees contribute across formal team boundaries. People can move toward a new company priority even when it does not fit their original reporting structure.

He says transparency helps employees make decisions for the wider project. Jensen’s “mission is the boss” idea gives teams a shared purpose while leaving room for people to act.

Liu says a strong team needs trust and complementary strengths, not a contest to prove who is smartest. People should be able to share ideas, challenge one another, and focus on what the group needs.

Asked whether his low-ego stance is genuine or a mask, Liu calls it confidence. He holds the work to a high standard but wants to build a foundation that helps others move faster, rather than beat competitors.

As a researcher, Liu once measured success by whether his paper beat other methods. Now he takes satisfaction in helping customers and partners become more capable, and in building a foundation other teams can use.

As a young researcher, Liu wanted every paper to outperform others. Over time, he found satisfaction in helping partners become more efficient and developing new talent, even after another model passes his benchmark.

Liu says a researcher’s old measure of success was beating other papers, but his priorities changed as he became a manager. He now values helping the next generation grow and seeing partners solve problems more efficiently.

He points to NVIDIA’s support for hardware partners, game developers, and software frameworks. The company contributed to ecosystems around PyTorch, TensorFlow, Triton, and vLLM because developers’ success expands the platform.

Liu calls Huang a “hexagonal warrior” who can discuss low-level computer registers, product pricing, marketing, and customer relationships. Huang’s direct criticism can be intense, but Liu sees it as a way to push people toward their best work.

Huang’s criticism often comes with encouragement, Liu says. The intention is to push people toward better work, though Liu recognizes that this style demands time and energy from the leader as well.

Liu says the phrase “quiet, baby” is used when someone makes excuses instead of focusing on the work. Huang expects people to meet a high bar, while the team should judge contributions fairly rather than assume that only the current number-one model matters.

Liu says Huang’s “quiet baby” remark left a lasting impression. When a result falls short, he believes researchers should solve the conditions that caused the gap instead of relying on excuses.

For Liu, a model may lead a benchmark briefly before another passes it. What lasts is the capability to build strong models and the contribution they make to customers, partners, and the wider field.

Liu says absolute number one is difficult to define. He hopes NVIDIA can build frontier models, but does not want capability to concentrate power in a few hands if that would create an unhealthy imbalance.

He says the lasting measure is contribution: people pass through life once, so he wants his work to have helped the world and to show that his time was used well.

Asked what he still wants to achieve, Liu says Physical AI has not yet succeeded. He hopes world models will become an important force that helps the field reach a practical breakthrough.

Liu says external measures such as paper counts, citations, and awards can never be won by everyone. He is more interested in whether he can be proud of his work and whether it helps others.

He says his focus has shifted from competing for recognition to helping Cosmos users and his team. Liu hopes his work has made a quiet but meaningful difference to the people he has worked with.

Liu praises the pace and openness of China’s AI ecosystem, naming DeepSeek, Qwen, Doubao, MiniMax, Kimi, Hunyuan, and Zhipu. He notes that Chinese firms continue to take interns and diffuse research knowledge, while some U.S. frontier labs have cut back on internships.

Liu says most researchers are driven by the pursuit of knowledge and truth, not by faith in one technology. Once a problem becomes better understood, he expects people to move naturally toward new questions.

Liu says large language models have delivered clear value, especially through coding agents. As researchers understand one problem better, he expects them naturally to move on to new questions.

He expects 2026 to bring major changes as model companies go public. Their success recognizes years of foresight and contribution, but technology will not stop; the leading companies of the future will differ from those of today.

Liu speculates that model capabilities will gradually converge, making the model itself less of a differentiator. Companies will need ecosystems, products, and services around the model, just as software companies differentiate even though many can now write software.

He compares models to traditional Chinese medicine: a black box can become a body of practical knowledge through many attempts. What matters is how a company uses the model, and new entrants can still catch up if they find a useful niche.

Liu says new teams may still catch up if they can mobilize talent and compute. If building a frontier model stops being a decisive advantage, companies may instead find mutually beneficial ways to use shared systems.

If building a model remains a decisive advantage, more teams will invest; if not, companies may find mutually beneficial ways to use shared models. Liu sees this as a natural balance.

Liu expects on-device models to grow alongside Physical AI but cannot predict exactly when. He believes world models will help physical systems generalize, while the field continues exploring modalities and deployment; a practical breakthrough is still ahead.

In the United States, relatively few companies build robot hardware, while China has many firms developing the robot body as well as software. Liu says China’s manufacturing base creates room for faster hardware iteration and tighter integration.

Liu says hardware ownership can give robotics companies more control over software-hardware integration. China’s manufacturing base may make rapid iteration easier, though the financial model still matters.

A Physical AI “ChatGPT moment” might arrive when a robot can learn a task from a simple demonstration and perform it immediately. Liu identifies generalization—from observation to action in unfamiliar scenes—as the central problem.

The sim-to-real gap is another generalization challenge. Cosmos can help with better data and better starting points now, while fully realistic training environments remain an important future goal.

Liu hopes that in five years they can name the precise date and place of the Physical AI breakthrough. He would like Cosmos to be a driving force helping several builders reach that moment.

In a quick-fire round, Liu names beef noodle soup and the Bay Area as favorites. He reads widely, likes Jin Yong’s martial-arts novels, and says Huang once recommended a psychology book about positioning and understanding problems.

Liu advises young researchers not to panic that AI will make their work obsolete next year. They should stay calm, choose worthwhile problems, and avoid letting hype make their options feel narrower.

He sees Physical AI as a promising area, but says AI’s impact will extend to materials, drugs, biology, and every industry. Researchers should look for places where their energy can make a meaningful difference.

Liu says model development should not be a Squid Game if teams are solving different problems. The world has many unmet needs, and different people can create value in different ways.

He says data and infrastructure often matter more than a clever algorithm alone. If data can solve a problem, collect or generate it; use algorithms for the parts data cannot handle, and iterate through disciplined experiments.

Asked whether every company needs its own large model, Liu compares models to electricity: companies build their own power plants only when it makes economic sense. If models become commodities, most firms will use shared services.

Liu says it is not easy for one group of researchers to stay perfectly aligned forever. People have different ambitions, and compute can be purchased by anyone with resources; companies can build models, but do not always need to.

Cosmos is not a war NVIDIA cannot afford to lose. Liu says NVIDIA wants to create a market for Physical AI: if households eventually buy multiple robots, demand for compute could expand dramatically and benefit many companies.

Huang often says to create markets instead of fighting over existing ones. A “zero-billion-dollar business” is not a business yet, but can become a billion-dollar market if the idea succeeds.

Liu says a “zero-billion-dollar business” does not exist yet; it becomes a billion-dollar opportunity only if the new market succeeds. He says NVIDIA’s willingness to take risks helped make CUDA possible.

Liu says NVIDIA’s bets are risky, as CUDA once was, but not taking them can be riskier: without programmability, GPUs might have become commodities. The interview closes with Huang’s motivation to take care of families and the fulfillment of changing the world through AI.