Steeve Morin on AI Inference, Chips and the Cost of Switching Hardware

ZML founder and CEO Steeve Morin joins Harry Stebbings to discuss AI inference, hardware economics and competition between model providers. ZML builds software intended to run models across different chips, giving Morin a direct commercial interest in the portability he advocates.

Morin describes ZML as an infrastructure layer intended to run models on hardware from different suppliers. Its promise is to make model deployment more portable without sacrificing performance, a claim about his own company rather than an independently verified comparison.

He expects AI applications increasingly to combine multiple models or backend components. A user may see one interface even when different systems handle text, images or other tasks behind it.

Morin argues that hardware choice can materially affect inference economics, citing an example in which an AMD configuration is more efficient than Nvidia for a particular 70-billion-parameter model. The magnitude depends on model, hardware, workload and cost assumptions; it is not a universal fourfold advantage.

He attributes Nvidia’s strong position partly to the combination of PyTorch and CUDA and partly to hardware availability. Developers can often reuse established tools on the GPUs they already have, while supporting another stack introduces engineering work.

Morin questions the economics of purchasing expensive H100 GPUs for inference and financing them over long periods. He describes a possible price and performance mismatch as a bubble risk, not a verified forecast of Nvidia’s financial results.

For interactive chat, output appearing quickly can matter; for agents and reasoning, he argues, the total time to complete a task may be more important. This changes how he thinks about throughput across many users versus latency for an individual request.

He describes GPU architecture as an adaptation of hardware originally designed for highly parallel graphics. Specialized AI chips can make different trade-offs, particularly around how quickly data moves between compute and memory; his comparisons are technical opinions rather than a complete benchmark.

For large-scale training, Morin emphasizes the interconnect connecting accelerators and credits Nvidia’s networking technology as an important part of its position. He says inference workloads can have different networking requirements, depending on how a model is deployed.

He divides available compute into GPUs that can be bought or rented, cloud-specific accelerators such as Google TPUs and AWS Trainium, and newer dedicated chips. In his view, users benefit from being able to choose among them rather than being locked into one supplier.

Morin discusses margins across chip manufacturing, accelerator vendors and cloud providers. The percentages he quotes are rough assertions and should not be added mechanically or treated as audited margins for a particular deployment.

Training and inference optimize for different things, he says. Training often prioritizes experimentation and rapid iteration across a large system, while production inference must deliver reliable responses at scale and control operating cost.

He compares making one painting with reliably producing a million: the tools and process may differ even though both involve painting. For inference, provisioning, monitoring and reliability become as important as getting a model to run once.

A major operational challenge is autoscaling: adding capacity when demand rises and reducing it when demand falls. Morin says unused accelerators can be expensive, but the savings depend on utilization and the deployment rather than a fixed multiple.

Although he sees technical opportunities for alternatives, Morin acknowledges that Nvidia chips are widely available and will remain in use for inference. The presence of deployed hardware is an advantage even if another architecture looks better on a particular metric.

He discusses Cerebras and Groq as examples of systems designed for high single-stream speed. He speculates about the cost economics of some offerings and explicitly says he does not have the underlying details; those comments are not verified loss figures.

Fast on-chip SRAM can reduce the time spent fetching data, but it consumes chip area and can raise manufacturing costs. Morin contrasts that with external memory and says a practical design must balance speed, capacity and price.

Over the next several years he expects reasoning and agents to increase demand for different latency characteristics. His discussion of latent-space reasoning and alternative architectures is a technical thesis about possible future systems, not proof that present models operate like human minds.

He expects both high-bandwidth memory and on-chip memory to remain important. More specialized architectures and compute-in-memory approaches may improve data movement, though their future cost and deployment are uncertain.

Asked to divide future AI compute value between inference and training, Morin estimates 95% inference and 5% training in five years. This is a speculative personal forecast; the interview does not define a measured revenue or spending series behind the split.

He argues software that abstracts hardware differences could make vendors compete on practical performance and cost rather than developer lock-in. ZML’s business is built around that proposition, so he has a direct interest in the outcome.

Switching providers is difficult because maintaining multiple software stacks and amortizing existing equipment create costs. A buyer may need a large enough commitment to justify the transition, while limited supply can favor the largest customers.

Morin’s proposed answer is to reduce the effective cost of switching so customers can choose available compute more easily. He describes working with chip makers to support their hardware; this is ZML’s commercial thesis, not evidence that switching is literally costless today.

Discussing Microsoft and AMD, he asserts that a particular inference configuration can fit more models on available memory and achieve higher throughput. He also claims a benefit for Microsoft’s inference economics; those company-specific purchasing and profitability claims are not independently established in the interview.

His example compares a specific model placement across different accelerators and does not imply that adding GPUs always gives the same incremental performance. Actual results depend on architecture, workload, parallelism, memory and software.

Pressed to choose a stock, Morin says he would choose Nvidia at that time because of supply, while saying his view might change if alternatives gain traction. He explicitly frames the remark as his opinion rather than investment advice.

He says data-center spending is still heavily directed toward training and that expanding an existing homogeneous GPU fleet can be operationally simpler than adding a second hardware stack. This helps explain why technical efficiency does not automatically determine procurement.

Morin uses a triangle of products, data and compute to assess AI competition. He sees Google as especially well positioned because of its distribution, data and internally developed compute; this is his competitive judgment, not proof of a future winner.

He views dependence on another company’s infrastructure as a potential cost and strategic disadvantage for model providers. Actual agreements and margins vary, so the interview does not establish the unit economics of every provider.

On scaling, he contrasts adding more compute with improving efficiency and exploring new architectures. He points to DeepSeek as an example of engineering under constraints and discusses Yann LeCun’s world-model work as a possible longer-term direction.

Morin is interested in models that reason in representations other than sequential natural-language tokens, and in diffusion-based approaches. He presents these as research directions, not proven replacements for current models.

He considers model distillation fair game, while the host asks about the legal and ethical implications. That is Morin’s opinion; the interview does not determine what any company did or whether a particular use complies with law or contract.

On synthetic data, he is uncertain. He notes that games and code can provide external feedback through rules or execution, while repeatedly training on model-generated text may introduce quality problems; he does not claim the answer is settled.

He expects smaller models to remain attractive for efficiency but allows that a more capable model with a large context window could sometimes replace fine-tuning. Which approach wins depends on future cost, accuracy and operational requirements.

Morin explains retrieval-augmented generation as retrieving relevant information and adding it to a model’s context before a response. The method can ground a request in external material, but retrieval quality and context limits still constrain the result.

He praises DeepSeek’s efficiency work and believes constraints can encourage innovation. He also notes the strength of OpenAI’s consumer brand, while cautioning that media attention around a new model does not by itself establish long-term competitive results.

Morin is not convinced that European regulation is the decisive obstacle for every AI startup and says he cannot evaluate all of Mistral’s finances. His remarks on those companies and regions are opinions made at the time of the conversation.

He is skeptical of the headline scale of the Stargate announcement but says he does not know whether the announced figure will materialize. He considers talent, energy and efficiency important constraints alongside the availability of capital.

His advice to AI startups is to pay close attention to how much of their revenue is consumed by model and compute costs. A differentiated product needs room to cover its own operations rather than merely reselling expensive compute, in his view.

Morin raises concerns about Nvidia Blackwell supply and engineering based on accounts he says he has heard. Claims about canceled orders, delivery practices and a possible downturn are unverified in the interview and remain his speculation.

He concludes that hardware supply, latency and future architectures could reshape inference, while acknowledging that his estimates may be wrong. The practical question for customers remains how to run a useful workload reliably at an acceptable cost.