Jensen Huang: For 25 Years, 99 Percent Were Right. NVIDIA Bet on the 1 Percent

Sarah Guo and Elad Gil recorded this episode of No Priors at NVIDIA’s Santa Clara headquarters. Guo had checked the market cap that morning: about $674 billion. The A100 is the workhorse of the current AI ecosystem, and the man who has run the company since 1993 describes it as something he is always trying to keep alive. Huang has been at it for 30 years. Making money, he says later, is a skill, and it took him a long time to learn. For the first 20 of those years he was still figuring it out.

The origin story is a minority bet. In 1993 the valley was split between general-purpose computing and accelerators. About 99 percent of the value, Huang says, was believed to sit in general-purpose machines. About one percent believed in acceleration. For 25 years, the 99 percent were right. NVIDIA started a company on the one percent anyway. The only honest use of accelerated computing at the time was problems that general-purpose machines could barely solve, or could not solve at all. That became the mission: solve problems normal computers cannot.

Follow that mission to its limit, he says, and you arrive at self-driving cars, robotics, climate science, digital biology, and the most famous case, artificial intelligence. The rest of the conversation is how a rejected architecture became the engine under ChatGPT, and how a CEO of a hundreds-of-billions company still talks about trying to do as little as possible.

Huang’s path into chips was almost accidental. At Oregon State University the computer-science lab had a poster of AMD’s 29000 32-bit processor, and he thought it would be cool to build one. On campus recruiting day two companies stuck. He joined AMD to design microprocessors. His office mate then went to LSI Logic, a startup founded by Silicon Valley legend Wilf Corrigan to design chips with software instead of by hand, and insisted he follow. LSI was at the beginning of the EDA industry, the moment every company could start building its own chips. He calls going there one of the best things that ever happened to him.

The people he met there were a roll call of computer architecture. Andy Bechtolsheim, founder of Sun. Architects from Silicon Graphics. John Rubinstein, then at Dana Computer, later Apple’s vice president of engineering. NVIDIA’s other two founders, Chris Malachowsky and Curtis Priem, belong to the same stretch. He learned to build computers out of chips very young. Starting a company, though, was not his idea. It was theirs. Chris and Curtis wanted to leave Sun. Huang was doing well at LSI, liked the job, and had two kids with his wife Lori. He told them he really needed to have a job. They kept hounding him anyway.

The three of them landed on accelerated computing because the rest of the industry would not. That choice, more than any product line, is what he still uses to explain why NVIDIA ends up in autonomous vehicles and large language models. If you spend your life on applications that are barely possible, he says later, people call you. You are the horse they call to solve those problems.

The technical problem of acceleration is a tightrope. Make the chip too general and you get less speedup in any one domain; become as general as a CPU and you cannot accelerate a CPU with a CPU. Stay too narrow and your R&D dollars cannot keep up with the CPU, which had the largest chip R&D budget on the planet. Huang’s arithmetic from the early years is grim. Graphics was about a billion-dollar market. NVIDIA was putting roughly $150 million a year into it. How do you keep up with a few-hundred-billion-dollar industry? It is not even sensible. You have to find the niche where $150 million buys insane acceleration, then grow the market from one billion to five to ten without falling off the cliff.

That is how the aperture widened: post-process image effects after a render, particle physics and fluid dynamics to bring a scene to life, then molecular dynamics and seismic processing. CUDA grew out of making the graphics processor more general while staying on the right side of that line. One decision looked thankless at the time. Every NVIDIA chip would stay architecture-compatible. If each chip were a snowflake, a developer who had learned CUDA still would not know which part to write for. If this was going to be a computing platform, it had to behave like x86 or ARM: every chip the same.

For the first five to ten years CUDA had very few customers, and they still made every chip code-compatible. Huang invites people to look up the old gross margins. They started poor and got worse. NVIDIA was already in a viciously competitive industry, still learning to build cost-effective products, and then they layered on an architecture nobody paid for and that had no applications. The thing that carried the platform into the world was GeForce, a gaming card. The first GPUs Geoffrey Hinton brought into his lab were GeForce boards. Elad, Huang says, can tell the scene: Hinton walking in with a couple of cards called GeForce and telling the group to try them on deep neural nets.

Huang dates the AI moment to around 2012, and it arrived from several directions at once. Andrew Ng reached out to NVIDIA chief scientist Bill Dally about getting a neural-network model onto GPUs, swapping a few cards for thousands of CPU servers. Almost simultaneously Geoffrey Hinton called, and the same thing was happening in Yann LeCun’s lab. Several labs showing a neural-network emergence at the same time is what got NVIDIA’s attention. ImageNet, and AlexNet, was the Big Bang. If it had only been computer vision, that would already have been enough for cars and robots. The insight that changed the company was larger: this might be a new way of writing software altogether.

ImageNet, with no specifically human-engineered algorithm, reached the effectiveness of 30 years of computer-vision work overnight, and not by a small amount. That forced a chain of questions. Why is it so effective? Will it scale? If it scales, what does that do to chip design, system design, interconnect, algorithms, and system software? Huang calls the neural network a universal function approximator: problems of extraordinarily high dimensionality, trained into existence one layer at a time if you have enough data. The first five years after that were about reasoning through the impact on computer science as a whole, while the models themselves marched from CNNs to ResNets to RNNs and LSTMs, with perception models getting larger.

Transformers, on first principles, looked like a big deal: an architecture that could learn patterns in spatial and sequential data, train in parallel, and scale. They overcame the limits of RNNs and LSTMs. NVIDIA trained some early language models, including BERT, and saw good results. ChatGPT still needed the last pieces: reinforcement learning from human feedback, retrieval models, and dialogue managers for guardrails. Once those snapped together you had the fastest-growing application in history, and a deeper claim. Computer programming, Huang says, has been completely disrupted. For the first time, the language of programming a computer is human language, any human language, and it does not even have to be grammatically correct. Anyone can program now.

He wants people to sit with a specific observation. Give ChatGPT a problem and ask it to reason step by step, and it may arrive at the wrong answer. Tell the same model to write a program to solve the same problem, and the program can run perfectly. One application that both reasons and writes a systematic solver is something whose implications you have to wrap your head around. Asked about machine sentience, he does not know what the word means technically. He is fairly sure he himself is sentient, less so on a hoarse afternoon. What he will confirm is that there is now software that can reason through many kinds of problems and provide a solution, or a program that provides a solution on an ongoing basis.

Will NVIDIA train more of its own models? Huang’s answer is a platform company’s answer. Go up the stack as far as developers need, and no further. What “developer” means keeps changing. Early on it was someone who controlled an operating system, so the company only had to reach device drivers. In scientific computing the developer is using a solver, and the algorithm is bound to the machine underneath. An algorithm written for CPUs talking over MPI is not the algorithm for thousands of processors on a fabric inside one GPU and thousands of GPUs in a data center. NVIDIA got good at redesigning domain algorithms: cuDNN is a domain-specific language for accelerated deep learning, RTX is the ray-tracing version of the same idea.

The developer of the future, he thinks, is someone who works with large language models and foundation models. If you can use ChatGPT, or OpenAI’s models through Microsoft, or Google, he really encourages that; they do an incredible job. The opening is elsewhere. Proteins, chemicals, climate-science multi-physics: those foundation models are niche enough that a horizontal consumer company will not build them, and the markets, drug discovery and climate tech, are still large. NVIDIA itself might build a foundation model for 3D graphics and virtual worlds, because those are super important to it, and one for robotics, because that sits at the intersection of things it already does well. Even then, take it as far as necessary and no further.

The word he uses for the boundary is lazy. “We’re trying to be as lazy as we can. Do as little as possible, as much as necessary.” Computer science’s first principle, as he recites it: reject work as quickly as you can; defer whatever is left as long as you can, until it can be rejected; do only what remains. Guo laughs that laziness as a company principle is the takeaway, then asks how that squares with CUDA, a decade-scale bet, and with activist-investor pressure when the company was worth a hundredth of today’s value. Investing in the future and staying alive now, Huang says, are not in conflict. The hard job is holding the institution’s core belief and being able to afford it. Conviction is one part. Making money is a skill, and a learnable one. He spent 20 years acquiring it.

Nobody, Huang says, should assume they are right for the CEO job. Gut-check that almost every day. What he never doubted is accelerated computing. By definition it is the way to solve problems that are impossible on ordinary machines, and if those applications later need to be broad-based, acceleration is still the right approach. When does the CPU hit its limit? For certain applications, 12 years ago. Hinton, LeCun, and Ng found that then. Computer graphics found it earlier. CPUs will always be needed. Someday everything will be accelerated, and for the applications we are all going to run, acceleration is the way forward. That conviction is why they started the company 30 years ago. He never believed it was wrong.

On management he is cheerfully heretical. Every org chart in the world looks like the U.S. military: someone on top, a pyramid coming down. CEOs have very few direct reports; first-level managers who just learned to manage have a crowd. That is the opposite of how it should probably be built. People who report to the CEO basically require no management. His own direct reports are sophisticated, talented, good at the job, excellent leaders with business acumen and vision. He has 40-odd of them. No one-on-ones. No career coaching. “What would you like to do with your life” is a conversation for new college grads and early-career people, and NVIDIA loves those conversations. At the executive layer the organization exists so they can pursue a lot of different things at once.

The looseness stops where architecture starts. A software company has to understand computer architecture, and you can only afford one. The largest company on the planet maintains two operating systems; too many firms keep seven, eight, nine instruction sets around. NVIDIA has one instruction set, one architecture, and is super disciplined about that. The company is tapered: discovery and innovation at the senior level, extreme focus where it must be focused. There is no generic company architecture. It should fit the work, the purpose, and the leadership style. NVIDIA actually runs two systems side by side. A large part of the company is built to invent and refine very complicated computers perfectly. Another part is Skunk Works, inventing ten-year-out things that may not work, pivoting, shape-shifting, killing investments that are not paying off. Those two ways of working have to sit next to each other.

Hopper’s two architectural ideas, as he tells them, are not a spec sheet. First, numerical format. Scientific computing uses 64-bit floating point; neural networks are statistical, so you can cut precision. Split 64 into eight and an AI supercomputer gets roughly 8x, nearly 10x across a couple of generations, just by admitting that 64-bit was unnecessary. Second, a Transformer engine: a pipeline shaped for training and inferencing Transformers because that architecture is so universal. The rest, he says, is brute force: the largest chip, the fastest chip, energy efficiency, the fastest memories, and connecting a bunch of them together.

The ten-year-out application he has confidence in is robotics. Autonomous driving is still making progress; he believes it will work. He also believes a robotic foundation model will be discovered: express yourself in human language and a mechatronic system with different kinds of limbs will figure out how to bend and articulate to do the task. What blocks that today? He has no idea. They will have to discover their way there. The road signs are visible. Models already learn structure from unstructured language and images. The next big thing is video. If a model can watch video and learn how humans articulate, it might generalize into a robot’s action system. His guess is less than ten years, probably about five, and then some pretty amazing robots. Google’s PaLM-E is a step on that Transformer path. NVIDIA also spent years on Ian Goodfellow’s GANs, style transfer, high-resolution image generation, and variational autoencoders, cousins of diffusion. If you can generate 2D and 3D images, he asks, why can’t you generate proteins and chemicals?

Advice for founders starts with a warning. Building a company is extraordinarily hard. He does not understand why anyone would do it twice. Gil, who started two companies, tries to talk second-time founders out of it. Huang compares it to having kids: there has to be a forgetting mechanism. Ignorance, he says, is one of the superpowers of an entrepreneur, and you will never have it again. Almost any advice would discourage you. If you are determined, do not wait too long. Go before you lose your ignorance. His own calendar is full of startup meetings inside NVIDIA: no momentum, starting from zero, painful every time, and rewarding when the thing gets used. The other attribute is a strange double belief. Be determined enough to stay with your conviction, and simultaneously believe you could be wrong, equally hard. He has watched talented startup CEOs who were almost right forget to be agile because they were so determined to be right.

Resilience is the same as forgetting. Do not dwell on the last point, the coach’s line, even when you just got your face kicked in. Crypto still makes his hands sweat. NVIDIA missed a quarter hard, going from no supply to too much, by about two billion dollars. Most CEOs miss by $15 million. The way he chooses what to do today is to walk ten years out, stand in the future, and look back at what he would have wished he had done, then do it now. Two industries make that list. Healthcare and drug discovery, a combination space larger than the number of atoms in the universe, where we can finally read the language and maybe the meaning of amino acids, proteins, and chemicals. Climate science, where estimates say you need a billion to a hundred billion times more computation than today’s fastest supercomputer; AI might cut that by a billion times. Earth-2 is the climate system. Clara is the medical one.

Guo’s last question, ranked with saving humanity and the Earth, is where the leather jackets come from. His wife. His daughter too. They hunt for jackets. Most of the ones hanging up are too fashion-forward for him to carry off; he picks the modest ones so he will not look out of place. A company then worth $674 billion still leaves jacket aesthetics to the family. That is, in its way, the same rule he gave for org charts: the structure should fit the leader.