Fractile’s Walter Goodwin wants to break the AI memory bottleneck

Walter Goodwin describes Fractile as a chip company building rack-scale systems for frontier AI. Its staff spans machine-learning research, manufacturing processes, thermal design and the mechanics of putting an entire rack together.

Harry Stebbings asks whether a chip company can succeed as AI businesses integrate more of their technology stack. Goodwin says Fractile plans to make money from chips and racks rather than competing with customers by selling models.

That business model still requires model expertise. Goodwin says his team researches how architectures and workloads may change, because a chip takes years to design and the wrong forecast can produce the wrong hardware.

Stebbings asks whether vertical integration is an attempt to escape falling margins. Goodwin agrees that labs and cloud providers want alternatives to expensive incumbent chips, but distinguishes common workloads from frontier capabilities.

Some tasks already reach an intelligence threshold where an open model is good enough, Goodwin says. Even there, providers can compete on the cost of producing the same useful result.

He argues that frontier models have not yet become commodities: the best models can still command margins, and advances at the frontier can later improve smaller models. He cites DeepSeek’s model and inference software as an example of efficiency gained by designing across layers.

Goodwin expects competent specialist cloud providers to become more alike as a basic compute service, although today it is not yet clear which will execute well. Stebbings notes that providers are adding model routing and inference services to differentiate themselves.

Goodwin says deploying a new chip platform remains a riskier bet than using established GPUs or TPUs. He believes specialized systems could deliver more than tenfold improvements on selected workloads, a claim he acknowledges comes from a competitor in that market.

He points to Fluidstack’s deployment of Google TPUs as an example of a cloud provider diversifying from a single GPU supplier. A provider that makes such a transition successfully may gain an economic advantage, he argues.

Goodwin praises Nvidia’s ability to produce general-purpose systems at scale. Fractile’s bet is narrower: anticipate the future workload and make inference much faster, even if that sacrifices some generality.

Specialization need not mean a small market, he argues. Large language model inference has become a substantial workload in its own right, making a purpose-built system commercially plausible.

Goodwin explains the memory wall through two chip specifications: arithmetic capacity, measured in operations per second, and memory bandwidth, the rate at which data can move to the processor.

He estimates that arithmetic capacity has increased far more rapidly than memory bandwidth over the past 25 years. For sequential token generation, he says, a model repeatedly needs its weights and context, so data movement can limit speed before the arithmetic units are fully used.

In his example, a large model producing around 80 tokens per second may be limited by how often its weights can be loaded rather than by its processor’s theoretical calculation rate. The exact result depends on the workload and system.

Goodwin compares external DRAM to a warehouse whose boxes must be carried to an office for processing. SRAM placed on the processor is closer and faster to access, but has much less capacity per unit area and cost than dense DRAM.

The desired combination is fast access and large capacity. Goodwin says Fractile is working on placing DRAM directly above the logic chip instead of alongside it, creating many short vertical connections across the chip’s surface.

Existing high-bandwidth memory already stacks DRAM dies, he notes, but is not simply placed on top of the logic die in the way Fractile proposes. The approach requires specialized 3D integration and manufacturing work.

Goodwin says the company’s goal is not to be defined forever by one memory technology. It expects to pursue this design over multiple generations but wants to change its approach if the underlying AI workload changes.

He contrasts conventional chips using similar high-bandwidth memory components with SRAM-heavy alternatives that can be fast but are capacity constrained. His proposed approach aims to combine bandwidth closer to SRAM with DRAM’s density.

Reasoning models and long-running agents make speed important because they generate many tokens in sequence. Goodwin argues that long contexts also require substantial memory capacity, so optimizing only for a quick chatbot answer misses part of the workload.

He says the underlying 3D integration technologies build on prior work, but applying them to memory stacking needs both a redesigned architecture and manufacturing partners willing to support a custom process.

Stebbings challenges the gap between chip startup marketing and production. Goodwin says Fractile has received test silicon from foundries and is conducting more trials; full product tape-outs are planned for coming months, not described as completed volume production.

Scaling is a separate challenge. Goodwin points to competition for foundry capacity, high-bandwidth memory and the partners that complete physical design and manufacturing handoffs.

Fractile keeps that physical-design and manufacturing interface in-house rather than using Broadcom as its back-end partner. Goodwin says direct relationships give it room to develop different building blocks and work with suppliers on custom stacking.

He contrasts that with a common platform serving several chip customers: their architectures can differ, but major manufacturing and memory components may be shared. A custom approach can create differentiation, though it also requires coordination with larger suppliers.

Goodwin believes using different manufacturing processes and memory generations may give Fractile access to capacity outside the most contested supply chain. He presents the possibility as part of the company’s production strategy.

Tape-out is the handoff of the completed chip design file to the foundry. The file specifies the physical layout used to make manufacturing masks; errors become expensive to correct after that handoff.

Tape-out is only the beginning of the next phase. After months in fabrication, engineers must bring the returned silicon up in the lab and run workloads on a chip, then a tray, and eventually a rack.

Fractile tracks a milestone it calls task zero: a reference workload running on a full rack of its chips. Goodwin says the goal is a reliable, manufacturable system, not merely a date for sending a design file.

Yield, redundancy, software and system reliability determine whether the product can be made in volume at workable economics. Goodwin thinks founders and investors sometimes focus too much on the visible tape-out date and too little on the work afterward.

The limiting resource in AI infrastructure has shifted over time, Goodwin says: first GPUs, then broader silicon and memory supply, and increasingly land, power and data-center capacity.

He argues that a fixed power budget makes efficient chips more valuable. A provider trying to serve fast responses on a limited number of megawatts must consider total useful output, not only the speed of one user’s response.

Goodwin says grid-connection queues indicate strong potential demand, while acknowledging that not every proposed project will necessarily be built. He sees power as a fundamental constraint on data-center growth.

Stebbings asks whether China’s energy policy gives it an advantage. Goodwin points to large hydro projects and argues that data centers can be located near remote power generation; he also sees opportunities to pair data centers with renewable power in the UK and Europe.

Goodwin’s original inference thesis was that training is a fixed cost and using the trained model must eventually justify it. He also anticipated spending more computation at inference time to improve answers, before reasoning models became widespread.

He describes two dimensions of difficult AI work: many agents can explore possibilities in parallel, while each may generate a long sequence of tokens. That combination can consume far more computation than a single chat response.

Given a difficult research problem, he says, a system could generate candidate solutions, search literature and compare alternatives over days. He presents inference compute as a way to spend more resources pursuing a better answer.

Stebbings questions whether financing can support that demand. Goodwin responds that model algorithms and hardware can both improve efficiency: for example, better ways of compressing context or selecting attention can extract more useful work from the same chip.

Goodwin argues that high GPU utilization and unresolved demand distinguish the current buildout from unused fiber after the dot-com boom. He expects improved models and hardware to deliver more useful computation per rack and megawatt.

Asked what a token will cost in ten years, Goodwin says the question depends on the model and capability behind it. He expects total spending on running models to rise, even as the amount of useful work obtained per unit of compute improves.

He sees durable value in both chips and frontier models. Chips have long design cycles and deployed lifetimes, while frontier models require expensive teams and computation and may command a premium for leading capabilities.

Company-specific fine-tuning can help today, he says, but an advantage can shrink when a stronger open model is released. He is uncertain how customized open models will compare with frontier services over time.

Goodwin questions whether software dedicated to orchestrating many different chip types will retain its value indefinitely. He thinks some diversity of hardware may be a temporary response to the limits of the available chips.

Using separate chips for different parts of inference creates a capacity-planning problem. Model weights can be shared across users, but each user’s context differs; changing model size or context length can leave a fixed mix of chips underused.

Goodwin argues that Fractile’s proposed stacked DRAM design could combine compute capacity, bandwidth and memory capacity in one system. This is the company’s architectural objective, not a demonstrated production benchmark in the interview.

Asked about Nvidia’s Groq deal, he says the Groq team and its fast-inference platform addressed a genuine need. He regards SRAM capacity and cost as limits for very long-context agents, while acknowledging the platform’s speed advantage.

He declines to choose among specialist inference providers but likes their ability to try new hardware before locking in massive owned capacity. He expects a small number of very large companies to emerge in inference serving.

Fractile raised a $720 million Series C. Goodwin says the capital lets it work on more chip generations, networking and systems in parallel, and make longer-term supply agreements.

The company will keep investing in model research to forecast changing workloads, he says. It is also expanding teams across Taiwan and the US and building expertise in rack-level systems and data-center integration.

Asked which supplier he would invest in, Goodwin chooses TSMC for its role in manufacturing leading chips. He also names D-Matrix as another company pursuing 3D memory technology.

Goodwin says recent capability advances have shortened his expectations for progress on verifiable problems such as mathematics and coding. His forecast that difficult verifiable challenges will be tackled within a year is his own prediction.

He predicts Nvidia could reach a $10 trillion valuation within five years and argues that demand for AI infrastructure will persist. He also says existing data centers must replace chips over time, creating opportunities for new entrants.

To strengthen the UK technology sector, Goodwin prioritizes expanding energy supply, including renewables and nuclear power. He ends with hopes that AI will help improve medical treatment and make education more personalized.