The New Chips Challenging NVIDIA’s AI Advantage

NVIDIA went from making graphics chips for games to supplying much of the compute behind today’s AI. Its hardware, CUDA software ecosystem and large commercial relationships make it difficult to displace. I think that dependence is motivating customers and countries to develop alternatives.

US export restrictions have made Chinese labs particularly motivated to use domestic hardware, including Huawei chips. OpenAI is developing its own chip as well. These projects are different in scale and purpose, but each offers a way to reduce reliance on a single supplier.

To understand the challenge, separate the chip from the software ecosystem. GPUs excel at highly parallel calculations used in model inference and training, while CUDA gives developers a mature set of tools. Hardware performance alone does not automatically replace that ecosystem.

Memory capacity and bandwidth also matter: a model’s weights need to be stored and read quickly. NVIDIA sells cards with different combinations of compute and memory, and I think its pricing reflects what customers who need both are willing to pay.

I compare the RTX 5090 with the RTX PRO 6000 Blackwell. The consumer card has 32 GB of GDDR7, while the professional card has 96 GB with error correction. At the prices I observed, the professional card costs much more despite comparable raw compute in some workloads; exact prices and performance vary.

Why not simply connect two cheaper GPUs and combine their memory? Imagine chefs whose extra refrigerator is in another building: capacity increases, but fetching ingredients takes time. Splitting a model can work, but communication bandwidth and how the work is partitioned can become bottlenecks.

Mixture-of-experts models and other techniques can help distribute work, but a connection between consumer GPUs is not equivalent to one large pool of fast local memory. The useful arrangement depends on the model, interconnect and software.

NVIDIA’s DGX Spark offers another tradeoff: 128 GB of unified memory and CUDA support in a small computer, at around $4,000 at launch. Its LPDDR memory has much less bandwidth than a high-end discrete GPU, and its compute capability is also different. I have been unhappy with my experience using it.

The Spark can connect to another unit over a high-speed network, though I found setup difficult. NVIDIA’s range thus offers relatively accessible routes to either more capacity or more GPU compute, while combining high capacity and high performance costs substantially more.

One reason I am revisiting the subject is Ox Alpha, the anonymous model later identified as GLM-5.3-Flash. SemiAnalysis reported that the service’s large volume of traffic was served on Chinese chips. I found it impressive that a useful model could be served at scale without an NVIDIA stack.

The same discussion includes OpenAI’s Jalapeño inference chip and its work with Cerebras. These are attempts to improve speed and cost and reduce dependence on NVIDIA, though they address a different setting from running a model on a desktop.

Apple also announced an M5 Max and M5 Ultra Mac Studio. The figures shown in the video include up to 512 GB unified memory and 1.2 TB/s memory bandwidth for the Ultra. Unified memory can be shared by the CPU and GPU, which is useful when a model needs more capacity than a consumer graphics card provides.

Apple claimed a major improvement in time to first token relative to an M1 Ultra system. This is a vendor comparison, and actual model performance depends on workload, software and configuration. At the time of the video the new machine had not yet been independently tested at scale.

The comparison on screen lists a 256 GB M5 Ultra configuration at $9,499, a 32 GB RTX 5090, a 96 GB RTX PRO 6000 and a 128 GB DGX Spark. They are not identical products: the Mac and Spark are complete computers, while the other two are graphics cards.

The Mac’s advertised memory bandwidth is much higher than the Spark’s and closer to the discrete cards, while it offers more capacity in the configuration shown. I think this could be a compelling local inference option, but bandwidth alone cannot establish end-to-end performance or replace CUDA compatibility.

I expect the M5 Ultra to make DGX Spark harder to justify for some local workloads. That is my assessment before release, not a measured result across every model. The NVIDIA card may still have an advantage in raw compute or software support, depending on the workload.

SemiAnalysis had early access to Jalapeño and reported strong inference results compared with tested NVIDIA, AMD and Google chips. The report describes an inference-oriented ASIC developed with Broadcom and emphasizes hardware–software co-design. The comparisons concern the specific models and setups tested, not every possible workload.

The specifications shown list 216 GiB of HBM, 700 W and 13.4 PFLOPS at FP4. The same table compares FP8 and memory bandwidth against several GPUs. A specification table is useful context, but differences in precision, system design and measurement complicate direct comparisons.

SemiAnalysis says Jalapeño can run several models, not only a single OpenAI model, and tested it with its InferenceX suite. The team also showed Doom running on the chip. Generality within the tested inference workloads should not be confused with evidence that it is a drop-in replacement for every GPU application.

The throughput-per-megawatt chart is striking. SemiAnalysis reports Jalapeño ahead of Blackwell on performance per watt in almost all of the scenarios it examined, across both lower-latency and higher-throughput conditions. Those claims remain tied to the benchmark’s configurations and power accounting.

The report describes over 700 tokens per second per user on DeepSeek R1 at concurrency one, using single-token prediction, and around 1,400 on the other models cited. It also reports comparable GSM-8K evaluation results. These are measured results in the reported setups, not a promise for all deployments.

HBM4 contributes to the chip’s memory bandwidth advantage over older memory generations. NVIDIA’s Rubin also uses HBM4. The availability and generation of hardware matter when interpreting comparisons between systems.

The report also notes a limit: the tested models were comparatively manageable in size. We do not yet know from those results how Jalapeño performs on the largest models or in every production deployment.

OpenAI is reportedly constrained by data-center power, making tokens per megawatt especially important. If the power budget is fixed, better throughput per watt can translate into more useful service from the same energy. Electricity is becoming a central constraint alongside chip supply.

In a CNBC clip, Jim Cramer asks NVIDIA CEO Jensen Huang about OpenAI building a competing chip. Huang says many chip projects start and some are canceled, and expresses confidence in NVIDIA’s technology, supply chain, market position and ability to support partners.

I see competitive pressure even if NVIDIA remains highly profitable. I also discuss a reported Hugging Face bid as a possible way to support open AI development and, in turn, encourage demand for NVIDIA hardware. That is my interpretation of the incentive, not proof of a completed acquisition.

Terafab is another proposed effort to expand chip manufacturing. I read its ambition as further evidence that large customers want alternatives and more control over supply. A proposed fabrication project is not the same thing as demonstrated production capacity or an existing NVIDIA replacement.

I should disclose that I own AMD stock. I have not focused on AMD here because I think it still has catching up to do, even though it could benefit from a more competitive market. That is my opinion as an investor, not a prediction of its performance.

I do not know which of these efforts will ultimately succeed. My broader argument is that more companies are trying to build alternatives, AI tools may accelerate development, and power will increasingly shape the economics. NVIDIA’s position is strong, but I do not think its lead is guaranteed forever.