Groq Founder Jonathan Ross on AI Compute, Chip Competition and Energy

Harry Stebbings interviews Groq founder and CEO Jonathan Ross about inference chips, model providers, energy and the economics of AI. Ross previously worked on Google’s TPU and now sells AI compute through Groq. His claims about demand, performance and competitors carry a direct commercial interest and are not independent benchmarks.

Asked whether AI is a bubble, Ross prefers to examine what major companies and governments are spending. He sees evidence of substantial value but describes returns as uneven, likening the market to early oil exploration: a few successes amid many failures. Spending by itself does not establish aggregate profits or future returns.

Stebbings presses him on escalating capital expenditure and whether it must eventually produce revenue. Ross agrees returns matter but says hyperscalers also spend defensively to preserve their position. He believes the aggregate financial return will be positive, while acknowledging that many individual investments may fail; this is a forecast.

As an example of AI value, Ross says a Groq team put a customer-requested feature into production in four hours using prompts rather than human-written code or human debugging. He imagines completing similar work during a customer meeting in future. The example is a company anecdote, and the future speed is speculative.

He says building competitive chips is difficult: multiple Google efforts occurred alongside TPU, and not all succeeded. Large companies may try several approaches to preserve options, but he does not expect all attempts to match established hardware. Ross helped build a TPU and now leads a competing supplier.

Stebbings asks whether proposed Nvidia investment in OpenAI, followed by chip purchases, creates a circular money loop. Ross argues that part of the spending goes to suppliers and real infrastructure, while Stebbings points to the remaining circular element. Their illustrative percentages are not a transaction audit and do not establish the terms or completion of a particular deal.

Ross wagers that doubling inference capacity for OpenAI or Anthropic could nearly double their revenue within a month. He cites rate limits and slower service as possible signs of constrained supply. Neither company supplies supporting financial or capacity data in the interview, so this is his hypothetical forecast, not verified revenue guidance.

Stebbings asks whether users necessarily need very low latency, noting that some may accept background tasks. Ross argues faster responses can improve engagement and draws on examples from consumer products and internet services. His numerical conversion claim is not supported with a study in the discussion, and latency requirements vary by task.

Ross predicts that OpenAI, Anthropic and hyperscalers will eventually build their own chips. He says success requires not just silicon design but software and keeping pace with model changes. His confidence is a prediction, not confirmation of any company’s plans or execution.

He recalls Google building AMD-based servers while Intel was stronger in that generation and says the option helped negotiate with Intel. The account illustrates his view that developing an alternative can have bargaining value even when it is not deployed at scale; it is his recollection.

Ross describes high-bandwidth memory and packaging as constraints on GPU supply and characterizes Nvidia as an unusually large buyer of HBM. His numerical production estimates and discussion of suppliers’ incentives are his industry analysis, not disclosed supplier contracts or independently verified capacity data.

In his view, making a chip can give a large buyer greater control over access, even if its chip is less capable than Nvidia’s. A modest improvement in a chip can affect the economics of a much larger system. Whether custom hardware actually improves supply or cost depends on the design and supply chain.

He says much infrastructure spending goes toward data centers with long expected lives rather than only designing chips. Stebbings asks whether commonly used three-to-five-year chip amortization is too long. Ross says Groq uses more conservative assumptions and expects to upgrade chips frequently; these assumptions are not a forecast of every operator’s depreciation policy.

Ross separates two decisions: buying and deploying a chip must justify its capital cost, while continuing to run installed hardware can make sense if it covers operating costs. He cautions that new, cheaper hardware could push old hardware below that threshold, and long contracts can complicate the decision. These are simplified economic tests, not accounting advice.

He acknowledges that older H100s still generate revenue above operating costs in the market as he sees it, which he attributes to scarce compute. That is an observation and interpretation rather than proof of a particular chip’s age, utilization or profitability across all operators.

Ross says a prospective customer sought five times Groq’s total capacity and that neither Groq nor others could meet the request. He uses the anecdote to argue that supply, as well as speed, matters to customers. The customer and its alternatives are not independently verified.

Groq’s proposed advantage, he says, is a different supply chain: initial LPUs could arrive about six months after an order, compared with longer advance commitments for GPUs. His lead-time comparison is a company claim, dependent on order size and supply conditions, not a guarantee.

Ross invokes the “hardware lottery”: researchers may favor model architectures that run well on available hardware. Incumbents benefit when the ecosystem designs for their chips, he says, while a newcomer must react quickly to changes. This is his interpretation of competitive dynamics.

Despite predicting custom chips, Ross expects Nvidia to keep selling GPUs because infrastructure demand is growing. He argues additional inference can increase demand for training and vice versa. The strength of that feedback loop and future chip demand remain uncertain.

Ross says more inference compute can improve some products and sees AI spending as different from conventional SaaS costs. Stebbings challenges whether efficiency improvements weaken the link between compute and quality. Ross responds that lower prices can open cost-sensitive markets; neither view proves that more compute always improves every task.

Ross contends that some Chinese models cost more to run than an OpenAI open-weight model, even when their market prices are lower. He distinguishes training costs, inference costs and selling prices. This is a vendor’s comparison without a disclosed common workload, model-quality benchmark or cost methodology; it should not be generalized to all models.

Stebbings asks whether subsidies could offset a cost disadvantage. Ross predicts the United States will have an advantage in supplying allies over the next two to three years because of compute and energy efficiency, while China can support its domestic market. These geopolitical and cost projections are uncertain.

He favors releasing older model generations openly to encourage adoption and lower ecosystem costs. He sees brand, distribution and usage as important alongside model quality. That is his competitive strategy assessment, not proof that any provider can retain users regardless of model performance.

Asked whether nuclear is the only way to meet AI energy needs, Ross says no: he also sees potential in renewables and locating compute near inexpensive power. He proposes expanding Norwegian wind alongside hydropower, but his figures are illustrative claims and the interview does not establish permitting, grid, environmental or land feasibility.

Ross also supports nuclear power and attributes reluctance partly to fear. Stebbings questions how quickly large energy projects can actually be built. Ross suggests private and international financing and hosting data near energy resources, but those proposals depend on regulation, infrastructure and local decisions.

He argues that countries need both energy and compute to control their AI capacity. Stebbings worries European governments may move too slowly; Ross predicts economic consequences if they do. His suggested data-hosting arrangements and warnings are proposals and forecasts, not demonstrated outcomes.

Stebbings asks whether European model sovereignty can be a competitive advantage. Ross says control over a model is not enough without sufficient compute and discloses Groq’s partnership with Mistral. His view of that partner and its infrastructure needs is commercially interested.

He expects Nvidia to sell the GPUs it can make even if Groq supplies more inference chips, arguing that inference and training reinforce one another. He says AI adoption surprised him because language made it easy to use. This is his view of the market, not a validated forecast of each supplier’s sales.

Ross identifies model data, algorithms and compute as interacting inputs and argues that increasing any of them can improve AI. He describes compute as the most predictable lever, although he acknowledges limits in data and synthetic data. The value of additional compute depends on model design and workload.

He predicts AI will create labor shortages rather than mass unemployment, through lower prices, reduced working hours and new industries. He offers coffee farming and supply chains as examples of possible efficiency gains. These are speculative economic outcomes, not evidence that prices, wages or employment will move uniformly.

Ross invokes past shifts from agriculture to other work to illustrate new occupations emerging. His historical employment percentage is not substantiated in the interview, and the analogy cannot establish the speed or distribution of future job creation. Stebbings notes that this outlook differs from predictions of widespread unemployment.

Asked about the Trump administration, Ross gives a positive view of its AI policy and mentions permitting. This is a political assessment by a company executive, not a comprehensive evaluation of the administration’s policies.

On vibe coding, Ross predicts coding will become a broadly used skill, like reading and writing. He recounts a coffee-shop operator who built an inventory tool with prompts and then fixed problems reported by staff. This is an anecdote and a forecast, not proof that generated software works reliably without review.

Stebbings asks whether low margins matter for fast-growing AI companies. Ross says a business eventually needs to cover costs and that margins can provide resilience during volatility, while high margins may attract competitors. He favors keeping margins low enough to give customers value while retaining the ability to sustain the business.

Ross recalls a finance candidate proposing higher prices to bring demand in line with supply. Stebbings says the proposal makes economic sense; Ross argues lower prices build customer trust and can increase volume. His belief that extra supply will always find proportional demand is a business thesis, not a proven demand curve.

Discussing expensive markets and concentrated technology valuations, Ross distinguishes underlying value from popularity. He says private-equity firms want inexpensive AI compute and views that as a sign of value. His account comes from a supplier trying to sell that compute, and it does not independently establish returns for those customers.

Stebbings asks whether a slowdown in leading technology companies could affect the wider economy. Ross agrees that markets can overshoot and downturns can hurt even good businesses, but says he cannot predict whether a downturn will occur in the next year. Long-term optimism does not remove short-term market risk.

Ross says abundant financing can make hiring harder when talented engineers start their own firms. Stebbings asks whether that itself is evidence of overheating. Ross agrees it is a challenge but says AI may also make teams more productive; he does not claim to know how the competing forces will resolve.

He praises Google’s technical culture and sees Gemini adoption as progress, but criticizes some consumer integrations as incomplete. Stebbings argues that OpenAI has gained distribution quickly. Ross agrees Google could be late and says he sees at least Google and OpenAI as competitors; these are subjective, time-specific assessments.

The speakers note that OpenAI also offers coding tools. Ross says Groq engineers have switched among tools as capabilities change, while enterprise contracts may encourage slower switching. Their experience does not measure switching costs or durability across the whole market.

Asked to choose between hypothetical OpenAI and Anthropic valuations, Ross says he would invest in both and believes both are undervalued. He imagines the AI labs and large technology firms growing in value together. These are speculative valuation opinions, not investment advice or a guarantee.

Ross says successful model providers naturally move into applications their customers once built. Groq, he says, has chosen not to build its own models to avoid competing directly with customers. He admits that this line could be a mistake if a customer eventually subsumes Groq’s business.

Ross says Groq raised $750 million at a valuation near $7 billion after initially planning a smaller round. These are funding and valuation figures, not revenue, earnings or cash returned to investors. He says the round could have been larger but he was sensitive to dilution.

He says Groq’s hardware sales have positive margin and that popular models on its newer chip also have positive margin. Some other model services, he says, cover operating costs but have less satisfactory capital economics; he declines to disclose exact margins. Positive unit margin does not establish company-wide net profitability or cash flow.

He says his goal is to keep Groq’s margin as low as possible while maintaining stability and preserving the option to raise prices when needed. This is the company’s pricing strategy, with actual results dependent on utilization, operating costs and capital recovery.

For the chip market in five years, Ross predicts Nvidia will still earn more than half the revenue while selling a minority of chips. He offers an illustrative split and attributes the difference to brand and pricing. Neither the market shares nor the resulting returns are known.

He says he would be surprised if Nvidia were not worth $10 trillion in five years and suggests Groq could possibly reach the same valuation. These are highly speculative statements from the leader of a competing company, not valuation guarantees or financial recommendations.

Asked about SRAM costs, Ross agrees that SRAM can cost substantially more per bit than DRAM. He argues that system-level model replication and memory usage, rather than the cost of one bit alone, determine the comparison. His illustrative thousands-of-chips example is architecture- and workload-dependent, and the interview provides no independent benchmark.

He says Groq operates 13 data centers and distributes models and workloads across locations. He describes different optimizations by geography and capacity. The operating count and performance implications are company claims at the time of the interview.

Asked where he could take more risk, Ross says Groq could increase supply orders, but a customer needing five times current capacity may still be unreachable with only double. He balances faster expansion against the dilution of raising more capital. Future orders and financial outcomes are uncertain.

In quick-fire questions, Ross disputes that Nvidia’s software is a strong lock-in for inference, while acknowledging its importance in training. He says 2.2 million developers had signed up for Groq; signups are not the same as active users or paying customers. His competitive assessment is a vendor’s view.

Ross says he would not start a chip company at that moment because developing and shipping silicon takes years. He describes Groq’s development experience and says the company is moving toward a yearly product cadence. Those timelines and success rates are his reported claims, not guarantees for future chips.

He says Groq has shifted from preserving many options toward concentrating on fewer priorities as the business matures. He expects AI providers to differentiate, citing Anthropic’s coding focus and xAI’s connection to a social network. His predictions about their trajectories are subjective.

Asked to compare Google, Microsoft and Amazon, Ross offers differing near-term and long-term views, while Stebbings challenges his interpretation of Microsoft’s OpenAI relationship. These are forecasts about companies and strategy, not recommendations to buy or sell securities.

Ross closes by comparing language models to a telescope for the mind: he hopes they will reveal more about intelligence over time. His optimistic analogy expresses a personal outlook, not an established account of what models understand or what society will become.