Jensen Huang’s Computex 2024 Vision: NIM, Blackwell and Physical AI

Matthew Berman presents highlights from NVIDIA CEO Jensen Huang’s June 2024 Computex keynote in Taiwan. The presentation spans accelerated computing libraries, Earth-2, inference microservices, digital humans, Blackwell systems, networking and robotics. Demonstrations and performance comparisons are NVIDIA’s presentations at the time, not independent validation of every claim.

Huang introduces a virtual-world montage as simulation built with NVIDIA’s Omniverse. Berman interprets it as entirely generated in real time; the selected footage itself does not establish the production method or real-time status of every element. Simulation can combine rendering, authored assets and AI, and is still computer-generated imagery.

Berman calls CUDA an important part of NVIDIA’s competitive position. Huang explains that accelerated computing is not simply a compiler switch: algorithms often need to be restructured for parallel execution. He presents NVIDIA’s domain libraries as a way to make that work accessible to developers.

Huang lists cuDNN for deep learning, physics-oriented AI libraries, Aerial for accelerated telecommunications and cuLitho for computational lithography. He says TSMC was moving cuLitho toward production to accelerate parts of chip manufacturing. These are vendor descriptions of tools and planned adoption in 2024.

He continues with Parabricks for genomics, cuOpt for optimization and cuQuantum for simulating quantum systems. He also mentions cuDF and acceleration for data-processing ecosystems. His claims about records and speed are presented as NVIDIA’s claims rather than universal results for every problem or setup.

Huang says CUDA alone is too low-level for many domain users, and compares specialized libraries to other software interfaces that make complex work usable. NVIDIA’s investment in those libraries is, in his view, central to supporting a broad range of markets.

Berman introduces Earth-2 as an effort to build a digital twin for weather and climate. Huang says the goal is to improve understanding of climate change and disaster risks, and presents a storm forecasting demonstration that explicitly discusses uncertainty in the storm’s path. A simulation is a model-based tool, not a guarantee of an accurate forecast or disaster prevention.

Berman interprets Huang’s argument as a shift from packaged software to AI models, and speculates that AI could increasingly build software. Huang describes generative AI as producing tokens representing text, images and other kinds of information. He frames data centers running these models as “AI factories.”

Huang extends the token analogy to weather, molecules, vehicle controls and robot actions. He argues that a model can learn patterns and generate outputs in these domains. The ability to generate an output does not by itself establish accurate physical understanding or reliable real-world control.

He compares an AI factory producing tokens with an electricity generator producing power, and predicts a new industrial market. Huang’s market-scale claims and industrial-revolution framing are his commercial vision, not independently measured economic outcomes.

Huang predicts that future computers will generate more responses and retrieve only necessary information, providing task-oriented capabilities rather than only tools. He suggests generated responses can be more contextually relevant and efficient. These are predictions and depend on workload, accuracy and implementation.

Huang introduces NVIDIA NIM, short for NVIDIA Inference Microservices, as a way to package pretrained models with the software needed to deploy them. Serving large models can involve distributing work across GPUs and managing throughput, uptime and resource use, which he compares to factory operations.

He describes containers combining CUDA, cuDNN, TensorRT, Triton, cloud-native scaling, monitoring and standard APIs. NVIDIA presents the package as simplifying deployment across supported environments. Berman’s clip shows Huang offering a Llama 3 NIM to try; model licenses, hardware requirements and enterprise terms still apply.

Huang introduces digital humans as a potential interface for interactive services, while acknowledging the difficulty of crossing the uncanny valley. The video presents examples such as design assistance, customer service and a healthcare scenario. It is a promotional demonstration and does not establish clinical safety or replace professional medical advice.

The digital-human demonstration combines speech recognition and synthesis, language models, facial animation and rendering of skin and hair. NVIDIA’s ACE packages components as NIM microservices; the 2024 announcement distinguished generally available cloud services from early access on RTX PCs. Berman’s assertion that every part of the clip was live-generated is not independently established by the montage.

Huang discusses a shift from manually labeled data to models trained on large collections and predicts further progress through multimodal inputs, simulated data and self-play. He expects physically grounded AI to learn from video and simulation. These approaches may help with robotics but do not automatically solve physical reliability or the simulation-to-reality gap.

Huang presents Blackwell as a platform for larger models. He describes two GPU dies connected by a high-speed link, a Grace CPU, a second-generation Transformer Engine for lower precision, security features, NVLink and reliability tooling. His demonstration shows a production board; the specifications are NVIDIA’s 2024 presentation.

He says the CPU can support checkpointing or context memory, and emphasizes the need to detect hardware faults as clusters grow. He also describes a decompression engine intended to speed data processing. Actual throughput and reliability gains vary with the configuration and workload.

Huang compares eight years of NVIDIA computing performance and claims about a thousandfold increase in AI compute. He presents estimates for a GPT-4-like training workload, claiming energy use fell from approximately 1,000 to 3 gigawatt-hours from Pascal to Blackwell. This is a vendor-modeled comparison under assumptions, not a measurement of OpenAI’s actual GPT-4 training energy.

For inference, Huang cites an even larger reduction in energy per token across generations. Those numbers depend on the modeled workload, numerical precision, system and utilization; they cannot be generalized to every token or used as an independent estimate of the real GPT-4 service.

Huang shows an air-cooled DGX system with eight GPUs and a modular liquid-cooled alternative. He describes connecting 72 Blackwell GPUs using fifth-generation NVLink and compares aggregate bandwidth, compute and power with a prior eight-GPU system. The rack is a networked collection of GPUs, even when software treats resources as one domain.

He explains why large models require fast communication among GPUs, describing an NVLink switch and copper spine. Huang says the interconnect saves power compared with an alternative design. His hardware specifications and savings are vendor claims and depend on the comparison.

To connect systems beyond a rack, NVIDIA offers InfiniBand and Spectrum-X Ethernet. Huang says AI training traffic differs from ordinary internet traffic because GPUs exchange bursts of partial results and progress can wait for the slowest arrival. Existing data centers may prefer Ethernet because of their established infrastructure.

Huang identifies remote direct memory access, congestion control, adaptive routing and traffic isolation as ways to improve Ethernet for AI workloads. He illustrates how lower network utilization can increase the effective cost of a large data center. His statement that networking becomes effectively free is an economic characterization, not a literal zero price.

Huang lays out Spectrum-X generations intended for increasingly large GPU clusters and predicts demand for much larger installations. He expects AI interaction and reasoning to increase inference needs. The future cluster sizes and usage patterns are forecasts made at the 2024 keynote.

Huang thanks partners adopting Blackwell and says NVIDIA intends a yearly platform cadence. He describes designing an integrated data-center system while selling modular components so partners can build different configurations. Backward software compatibility, he says, helps the ecosystem carry prior investments forward.

He names Blackwell Ultra and reveals Rubin and Rubin Ultra as future platform codenames, describing them as under development. These were roadmap statements in June 2024, not proof that products were already available then.

The next section concerns physical AI. Huang predicts robots in factories and everyday products will need to perceive the world, interpret instructions and plan actions. The video shows examples of simulated learning; the presentation is a vision and selected demonstration, not evidence of general autonomous competence.

NVIDIA’s video describes reinforcement learning in simulated environments, where robots can practice actions and receive feedback before deployment. It presents Omniverse as a virtual-world platform combining physics simulation, rendering and AI. Simulation can reduce some costs and risks, but transfer to real environments still requires testing and validation.

The presentation describes three computing roles: systems for training AI models, Jetson computers for running models on robots, and Omniverse for simulated practice. NVIDIA says its software, libraries and models can be used selectively. The three-part architecture is a development framework, not a guarantee of safe deployment.

Huang introduces factory digital twins and says Foxconn and other Taiwanese manufacturers are working on them. In the demonstration, a virtual factory combines facility and equipment data from tools including Siemens and Autodesk. Planners can experiment with layouts and camera placement before making physical changes.

The Foxconn demonstration also shows simulated robotic manipulation and mobile-robot perception using Isaac technologies. NVIDIA describes testing object identification, inspection, trajectories and navigation before deployment on Jetson systems. The demo illustrates a proposed workflow rather than providing independent measurements of its effectiveness.

Huang explains how a factory’s virtual environment and a robot’s simulation can share a space. He highlights integration with Siemens and lists companies working with Isaac Manipulator or Perceptor for factory handling, logistics and inspection. These are announced collaborations and intended applications, not proof that every integration was operating at scale.

Huang says NVIDIA’s automotive stack was planned for future Mercedes-Benz and Jaguar Land Rover fleets and then turns to humanoid robots. He argues that human-shaped robots may benefit from environments and demonstrations designed around human bodies. The vehicle timelines and robot adoption are forward-looking statements from 2024.

Huang closes with a stage display of humanoid robots and predicts that manufacturers will build computers that walk and roll. Berman says he will continue following NVIDIA’s developments. The stage presentation shows hardware and the company’s direction, not a test of autonomous performance.