Jensen Huang: NVIDIA Is No Longer Building Computers. It Is Building AI Factories

A year after their first taping, Sarah Guo and Elad Gil are back at NVIDIA headquarters with Jensen Huang. The introduction now has to mention a market cap over $3 trillion and a company “literally holding all the chips in the AI revolution.” Thirty years into NVIDIA, they ask what bets still have a 10-year horizon: is it all scale from here, or are the architectures hitting limits on compute and memory?

Huang steps back before he answers. Computing has gone from coding to machine learning, from software written by people on CPUs designed for human coding to GPUs designed for machine learning. The whole stack changed, and with it the scale of problems that can be addressed. Parallelize software on one GPU, he says, and you have set the foundation to parallelize across a cluster, multiple clusters, or multiple data centers. That is the beginning of software and computing at a level nobody had imagined.

Over the next 10 years, Huang’s hope is to double or triple performance every year *at scale*—not at the chip—and therefore drive cost and energy down by a factor of two or three, every single year. Compound that for a few years and the curve is aggressive. Moore’s law was roughly 2x every couple of years. He would not be surprised to be on “some kind of a hyper Moore’s law curve,” and he fully hopes they continue to do that.

Why faster than Moore, which was famously self-fulfilling? The two technical pillars, Dennard scaling and Carver Mead’s VLSI scaling, were rigorous and have run out of steam. The new wave is co-design. Unless you can change the algorithm to reflect the system and the system to reflect the software, going back and forth, you have no hope. If you control both sides, you can walk precision from FP64 to FP32 to BF16 to FP8 to FP4 to “who knows what.” The second part is data-center scale: treat the network as a compute fabric and push work into it. That is why NVIDIA bought Mellanox and started fusing InfiniBand and NVLink so aggressively. Hundreds of GPUs now work together as what appears to be one incredible processor.

Inference-time scaling adds a contradiction. Before an answer, a model may tree-search, chain-of-thought, simulate in its head, prompt itself, and still need to respond in about a second—so latency must be extremely low. The data center, meanwhile, is still a factory that must emit high-throughput tokens to keep cost down and generate a return. Low latency and high throughput are at odds. To be great at both, NVIDIA had to invent something new. NVLink is that invention: a virtual GPU with enough FLOPs and working memory for context, and still enough bandwidth for token generation at the same time.

The model builders are compressing too. Elad cites a teammate’s number: over about 18 months, the cost of a million tokens into a GPT-4-equivalent model dropped 240x. On NVIDIA’s own layer, a solid CUDA foundation is what lets everything above it change. If the foundation moves, you cannot build a building. Since Llama first came out, they improved Hopper’s performance by a factor of five without the algorithm or the layer on top ever changing. A 5x in one year, Huang says, is impossible with traditional computing. Accelerated computing plus co-design made it inventable.

Infrastructure is already disaggregated. Sam Altman had just told Huang that OpenAI decommissioned Volta; they still have Pascals, Ampères, and Blackwells coming in air-cooled and liquid-cooled mixes. Services will have to take advantage of all of it. NVIDIA’s advantage, Huang says, is that what you build for training today will be wonderful for inference tomorrow. He believes most of ChatGPT is inferred on the same type of systems that trained recently. You leave a trail of infrastructure that is good at inference, recycle the return into the next training cluster, and trust that NVIDIA and the ecosystem will keep improving the old gear—another 5x in a year is the motion that “will never change.”

Inference itself will be multi-scale. Distilling a smaller model wants a larger teacher; frontier models will still do groundbreaking work and synthetic data. The market will run from giant models down to little tiny ones, not as generalizable but “quite effective” at one task, even superhuman in a tiny domain. He floats a name on the spot: not a small language model but a tiny language model, a TLM. All sizes, “kind of like software is today.” The expensive part of computing has not changed: you maintain software for as long as you shall live, you do not want to write it twice, and you want it to run on as large an installed base as possible. Tomorrow’s AI should run on today’s fleet; today’s software should run better on tomorrow’s hardware.

Guo notes the unit NVIDIA supports has grown from chip to server to rack-scale NVL72. What is next—the full data center? “In fact we build full data centers,” Huang says. They do not build PowerPoint slides and ship chips. Until the whole data center is stood up, you do not know the software works or the fabric delivers the efficiencies on the slide, which is why real performance is so often dramatically lower than peak. “The new unit of computing is the data center.” They build every combination: air-cooled, x86, liquid-cooled, Grace, Ethernet, InfiniBand, NVLink, no NVLink. The company has five supercomputers today and will easily build five more next year. “If you’re serious about software you build your own computers.”

The strange part is what happens next. They build at scale, vertically integrated, full-stack optimized—then they disaggregate everything and sell it in parts. The reason is grafting that architecture into GCP, AWS, Azure, and OCI, whose control planes, security planes, and cluster sizing all differ, while still making CUDA ubiquitous. Developers should write once and run everywhere, modulo about 10% because each cloud is optimized differently. That is a software principle NVIDIA “protect[s] quite dearly.” Software is the expensive investment. They have never given up on a piece of it. CUDA is used, he told people, because they will maintain it for as long as they shall live. NVIDIA Shield, the Android TV shipped seven years ago, still got a software update the week of the taping and a review calling it the best Android TV. GeForce has 300 million gamers; they have never stranded a single one. Without architectural compatibility, the software teams would have to be 100 times the size of the company.

The concrete case is the supercluster for xAI. Huang gives Elon Musk the first credit: pick the site, bring power and cooling, and decide to build a 100,000-GPU supercluster, the largest of its kind in one unit. They planned backwards from a stand-up date locked in months earlier. Every component, OEM, system, and software integration was staged. Network configurations were simulated as a digital twin. The supply chain and wiring were pre-staged. They even stood up a small “system zero” as ground truth before the rest arrived. When the tonnage showed up, gargantuan teams crawled over each other wiring 24/7, and within a few weeks the clusters were up. Huang calls it a testament to Musk’s willpower on mechanical and electrical obstacles that looked extraordinary.

The blocking problem, he says, was simply the tonnage of electronics. Usually a supercomputer of that class is planned for a couple of years, and from first delivery to serious work, “don’t be surprised if it’s a year.” They could not afford that. A few years earlier NVIDIA had an internal initiative called data center as a product: they do not sell it as a product, but they treat planning, stand-up, optimization, tuning, and operations as if they did. The goal is closer to opening a new iPhone. Give him space, power, and cooling, and they will help set it up in “call it 30 days.”

Looking ahead to 200,000, 500,000, a million GPUs, Guo asks for the biggest blocker. Huang’s list is capital, energy, supply—“everything.” Nothing at those scales is normal. Nothing is impossible either: no law of physics is in the way, but everything will be hard. Is it worth it? Getting to a computer that does what we ask, general intelligence or even close, would be a miracle. Five or six teams are climbing: OpenAI, Anthropic, xAI, Google, Meta, Microsoft. The next couple of clicks up that mountain are vital. “The prize for reinventing intelligence altogether is just too consequential not to attempt.”

A year earlier Huang had said extreme customers, especially in science, pull NVIDIA toward the next application. He still loves that, and he now loves something in-house: NVIDIA has AI chip designers and AI software engineers. How good are the chip designers? “Super good. We couldn’t build Hopper without it.” They explore a much larger space because they have infinite time on a supercomputer. Human engineers have so little time that they cannot search as much of the space, and they cannot search combinatorially—your exploration plus mine. The chips are so large they are designed almost like a thousand chips, each optimized in isolation. You really want cross-module co-design. AI can find local maxima hidden behind local minima. “You can’t do that without AI engineers. Just simply can’t do it. We just don’t have enough time.”

Elad then puts the stock on the table. When they last spoke, NVIDIA’s market cap was about $500 billion; it is now over $3 trillion. In 18 months the company added $2.5 trillion-plus, which he analogizes as a Stripe plus a bit, two and a half Snowflakes, “a country or two.” Walking the campus, he felt a buzz he remembered from Google 15 years ago. What changed in how NVIDIA functions, or in the size of bets Huang will take? Huang is dry first: “Our company can’t change as fast as a stock price.” In a lot of ways they have not. The thing to do is ask what is actually happening. Computing has been reinvented for the first time in 60 years. The marginal cost of computing is down probably by a million-x in 10 years, to the point where you can let the computer exhaustively write the software. Chip design is the same wish: let the computer discover something about the chip you could not. Digital biology is the same wish.

Then the industrial reframe. Classic data centers are multi-tenant stores of files. The new buildings are not that. They tend to be single-tenant. They are not storing files. They produce tokens, reconstituted into what appears to be intelligence: robotic motion, sequences of amino acids, chemical chains. “We’ve created a new instrument… the noun of the adjective generative AI.” Instead of the adjective, it is an AI factory, generating AI at extremely large scale. On one layer, a trillion dollars of infrastructure needs to be modernized because computing was reinvented. The bigger layer is a new commodity. How large can that industry be? Hard to say, “but it’s probably worth trillions.” NVIDIA does not build computers anymore. It builds factories. Every country will need one; every company will need one. “Give me an example of a company or industry that says, you know what, we don’t need to produce intelligence, we got plenty of it.” Someday people will say the semiconductor industry was never really about chips. It was the foundational fabric for society.

Embodiment, Huang says, is close in the same way AGI is close. Tokens are tokens. Tokenizing is not easy, but if you can tokenize, align with language models and other modalities, and generate a video of Jensen reaching for a coffee cup, why can’t you prompt a robot to generate the tokens that pick it up? Intuitively the problem statement is similar. The two brownfield robotic systems—meaning you do not have to change the environment—are self-driving cars with a digital chauffeur, and embodied humanoid robots. The world was already built for those two things, so you can bring robotics into the world without rebuilding the world. It is probably not a coincidence that Musk is focused on both: they are the forms with the largest potential scale.

The digital version is equally exciting. There is no question there will be AI employees of all kinds. NVIDIA’s outlook is some biological staff and some artificial intelligence, prompted the same way. Huang already prompts employees by giving context, assigning a mission, watching them recruit teammates, and iterating. That will not be different for AI marketing people, chip designers, or supply-chain staff. He hopes NVIDIA is someday biologically bigger and, from an AI perspective, much, much bigger. Asked which part of the company will be most artificially intelligent in a year, he picks chip design: start where it moves the needle most, on an insanely hard problem.

He can already imagine renting specialists. Sassine Ghazi at Synopsys and Anirudh Devgan at Cadence could have AI engineers, trained to be incredibly good at a particular module of their tools, that Huang might rent a million at a time, then rent a million from the other. Christian at SAP and Bill at ServiceNow will do the analog on their platforms. People say SaaS will be disrupted. Huang thinks the opposite: they are sitting on a gold mine. There will be a flourishing of agents specialized in Salesforce Lightning, in SAP’s ABAP, in CUDA, in Omniverse’s OpenUSD. Who will create an agent that is awesome at OpenUSD? NVIDIA, “because nobody cares about them more than we do.” Platforms will flourish with agents, introduce them to each other, and let them collaborate on problems.

What is underestimated? Huang’s answer is the underwater activity: groundbreaking science and engineering already being rewritten by AI and machine learning. You cannot walk into a science department or a theoretical math department, he says, where this work will not transform what they do tomorrow. Take every engineer and scientist working today as an early indication of the future, and you will see a tidal wave of generative AI and machine learning change everything in a short period of time.

He has seen this movie. In Toronto he watched Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, plus Yann LeCun and Andrew Ng at Stanford, and extrapolated from detecting cats—AlexNet, a toy by today’s standards—to a profound change in computer science. From that toy to superhuman object recognition was only a few years. Now the groundswell is in every field of science, none left behind, from quantum computing to quantum chemistry, already two or three years in. Give it another two or three, “the world’s going to change.” There will not be one paper, one breakthrough in science or engineering, “where generative AI isn’t at the foundation of it.” He is fairly certain.

Every so often someone asks if this is a fad. Go back to first principles. The computing stack changed. The way we write software changed, and that is core, because software is how humans encode knowledge and algorithms. Encode it a different way and nothing else will ever be the same. “Nothing will be left behind. We’re going to take everybody with us.” Guo adds the computer scientist’s thrill: at a robotics conference, a materials conference, a biotech conference, the driving of discovery is the same general algorithms, some universal unifying concepts. Huang is using it himself every day. It is his tutor. He does not learn anything without first going to AI—ChatGPT, sometimes Perplexity depending on how the question is formulated—then forking deeper. Even what he considers ground truth, even where he is the expert, he double-checks. Almost everything he does, he involves it.