A Dinner-Plate Chip Cut Training From 60 Days to 3.5

The GPU crunch is no longer a rumor at the edge of the industry. It is delaying training runs and product launches for generative-AI companies large and small. Sarah Guo and Elad Gil open this No Priors episode with that shortage, then bring on Andrew Feldman, CEO and co-founder of Cerebras Systems, which is building what they call one of the largest computer chips and fastest purpose-built AI processors ever made. Feldman is a rare serial founder in semiconductors. His previous company, SeaMicro, pioneered energy-efficient, high-bandwidth microservers and was acquired by AMD in 2012.

The week of the taping, Cerebras had just announced a strategic partnership with the UAE technology group G42: a $100 million deal to stand up nine AI supercomputers. Each machine is four exaflops of AI compute, for 36 exaflops in total. Feldman does not dress it up. When you find a partner who shares the vision and wants to build with you, and you get to start building the biggest computer on Earth, he says, it does not get better than that.

The company did not start from a 2023 slide about ChatGPT. In late 2015, five former colleagues began meeting regularly — Feldman notes they were in Guo's offices at the time — looking for the next computer to build. One day their CTO, Gary Lauterbach, leaned back and asked why a machine optimized for pushing pixels to a monitor would be ideal for AI. Then he added the line that stuck: would it not be serendipitous if twenty years of optimizing a part for one job left it well suited for another? They looked hard at GPUs and at the AI workload. By early 2016 they had decided they could build a better part, and the strategy was explicit: not a little better, but vastly better.

That bet sent them at wafer-scale integration, a technology that had never worked. The chip is roughly the size of a dinner plate; most chips are the size of a postage stamp. They chose it because they believed the workload would go very large, and that memory bandwidth plus the pain of breaking work across lots of little machines would become the binding constraints. This is Feldman's fifth startup. It was the first time, he says, that he undershot the market. In early 2016 they went out with eight pitches and came back with eight term sheets.

Guo had pulled her notes from that end-of-2015 meeting. Feldman had a slide arguing that the top seven problems in deep learning were all long training times — the whole industry bottlenecked on compute, before transformers had scaled. He jokes that she should send the snippet to his wife. More seriously, he treats the later success on transformers as an architectural test. They laid the design down before transformers existed, and he says they are now the fastest at transformers by a lot. When a design meets work it has never seen and is still really good at it, that is the sign you got the hardware architecture right.

Gil asks how they score the chip against GPUs. Feldman is not a believer in hand benchmarks. At AMD he watched a team of 30 whose job was to game them; when Lauterbach was at Sun, the benchmark team had 70 people. The real answer, he says, is how long it takes the customer to train a model — usually a collection of models. If you are aiming at a medium-size 13 billion parameter net, you almost always begin at 100 million parameters, then sweep 300 million, 600 million, maybe a billion, hunting for optimization points. How you do across that whole ladder is the score.

The clock that matters starts when the customer's model arrives and stops when the run finishes, including the time spent turning it into distributed compute. Look at the acknowledgments on large-model papers, Feldman says, and you will often see 20 or 30 people credited for tensor and model parallelism. On Cerebras equipment that step goes away, because the systems run strictly data parallel. That is time the customer gets back to train models instead of figuring out how to split a giant matrix multiply across a pile of GPUs.

The chip is a dataflow architecture made of about 850,000 identical tiles. Each tile is both processor and memory, so the machine is fully distributed-memory: SRAM sits one clock cycle from the core that uses it, which is why Feldman claims there are never memory-bandwidth problems. Cores wait for a token, or a flit, that tells them what to do and where to send the result, then push work forward. He argues that shape matches AI, which is itself a flow. The idea is older than the company. Dataflow was developed at MIT in the 1980s by Arvind; there was no good workload, so it sat unused until this one arrived.

Keeping a huge amount of SRAM on the wafer has a second payoff: they can harvest sparsity, something Feldman says others really struggle with. Parameters do not live on the chip. They sit in an off-chip parameter store and stream in. The result is strictly data-parallel training. Even in a 64-node cluster, every machine runs the same configuration, sees the same weights, and reads a different slice of the dataset. Scaling is linear, which he calls unheard of in computing.

That design is a direct answer to a GPU limitation customers hit every time they buy a card. Traditional GPUs fix memory on the interposer: you can have 40 gigabytes or 80. Want 120? Want 20? Want a bigger parameter store? You buy more GPUs, and you pay for compute you did not want or memory you did not want, because the two are tightly coupled. Cerebras disaggregates them, a trick Feldman says comes from supercomputing. You can run an arbitrarily large network — even a trillion-parameter network — on a single system. It would be slow, but you can run it, identify layers, and debug. On a GPU you cannot, because there is nowhere to store the parameters.

The G42 partnership is already showing up as models, not only as flops. Feldman points to BTLM, which he says is the number-one three-billion-parameter model on Hugging Face, with more than a million downloads and Fortune 100 companies already running internal work on it. That week they were set to open-source, with G42's Inception and MBZUAI, the largest Arabic LLM. Why get into training and releasing open-source models at all? They had so much compute that it was the cleanest way to prove how easy training on Cerebras is. In March they put seven GPT-style models into the open-source community while everybody else was putting one. Redistributing work across a GPU cluster is a large project; on their machines, he says, it is one keystroke.

Customers are also arriving with odd, specific requests. One asked Cerebras to design a three-billion-parameter model that could be pruned and quantized to run off a cell phone. Looking forward, Feldman does not assume models keep marching north of 175 billion. A network that size is expensive to train, and maybe more important, really expensive to serve: 17.5 times more expensive than a 10 billion parameter model. A lot of production models, he thinks, sit at 13 billion and smaller as an effort to keep serving costs down. Cerebras is training a collection in the 30-to-175 billion band, on the bet that band will still have demand.

He also expects a spike in multilingual models. Stability had just put out a Japanese one; there was a Spanish model; Inception, MBZUAI, and Cerebras were putting out Arabic. The hard part is data. Most giant datasets are internet scrapes, and most of the internet is English, so there are not enough tokens in other languages. Every nation, he says, wants its language and culture captured fairly in an LLM. Language carries experience — the New York experience includes Yiddish — and if you overwhelm a minority language with English, those nuances get sifted out. The joke on the show is that a Yiddish LLM would complain all the time, and that a Chinese one has the same mother. Under the joke is a real constraint: Arabic, Hebrew, and Hindi characters sometimes take two bytes rather than one, so a careless tokenizer further biases the model toward English. That, Feldman says, is exactly why G42 cares. They want the nuances of their culture reflected, not washed with 99 percent English tokens.

Guo asks what it is actually like to start a chip company, an uncommon experience even among well-funded founders. Feldman starts with structure. Venture investors love SaaS because a cheap prototype tells you whether customers like the product. Chips are the opposite. You spend two, two and a half, sometimes three years and $50 or $60 million building the first part, and only then do you learn whether people like it or hate it. There is a huge gaping chasm between starting and hearing from customers, a structural risk software companies do not carry. Software firms often spend as much money in the end; they just get the insight much earlier.

Engineering management is different too. On a two- or three-year project you cannot run 150 one-week sprints. Chip companies often staff three or four verification or QA engineers for each developer, because a bug can cost $20 or $30 million. There is no "we will fix it in 4.1.1." A serious bug means you respin the chip. The commitment to tooling, QA, and simulation is a different order of magnitude from a software shop.

That does not mean Cerebras is a hardware company in the old sense. About 75 percent of the work is software. The chip is the muscle; the software is the brain. They still need low-level code that runs against the hardware, and a compile stack so a user can write in PyTorch and run as if the machine were a GPU. A systems company ends up hiring many different kinds of engineers, not only silicon designers.

Gil wants a map of the accelerator market: who buys, and why would they buy anything but Nvidia? Feldman takes his hat off first. Nvidia has made hay while the sun shone and done an extraordinary job. Then the turn: they are now, in his words, extorting customers — extremely expensive, and unable to ship. That combination has opened the door for every alternative, including Cerebras. Nobody likes being dependent on a vendor. He suggests asking the people at Google and Facebook how they felt after years of depending on Intel. They disliked it intensely. The live tension is between that dislike and the need to keep running forward.

Large enterprises remain a solid part of the business. For smaller companies, a bit of consulting to accelerate model deployment is how you get in the door; once you show them how much easier it is, the order follows. He tells the story that opened the Chinese write-up of this episode. A customer trying to train on a GPU cluster had been at 60 days without convergence. Cerebras stood the job up, and three and a half days later the model converged. Customers can try before they buy on an internal cloud, starting with a small run. The roster, he says, runs from large enterprises and generative-AI startups to the military and intelligence communities and the supercomputing world.

How much will the silicon layer specialize — inference versus training, transformers versus diffusion, multimodal versus not? Feldman reduces it to the oldest questions in computer architecture: what you make special, what you make general, and what you choose not to be good at. Circuitry for transformers is not free; you pay a penalty every time you run a convolutional net, because idle specialty logic wastes power and area. Cerebras does not specialize at that layer. Every problem is linear algebra, predominantly sparse linear algebra, so they focused underneath the transformer. He does expect different silicon for training and for inference. Basic classification looks like the first pass of a training run; generative inference does not, because you have to hold far more information at the inference point. People using eight H100s to serve a big model are looking at half a million dollars of hardware.

The trick, as he frames it, is to solve hard problems in a general way so you are not depending on product management to guess the next fashionable layer type. If the field moved to single-shot learning that did not need much data, Nvidia would be out of luck — and so would Cerebras. Current machines are designed for extremely data-intensive, transformer-based work. Convolutional nets, transformers, diffusion, and things not yet invented will keep coming; they have not even run through the ideas people wrote down in the 1980s. Customizing a chip for one model type is not his preferred approach.

The supply crunch, Feldman says, comes from a profoundly inflexible chain. He calls a TSMC fab a pyramid of modern society: a $20 billion building in which nobody works, ideas go in, and $100 chips come out. Amazing, and unable to turn on a dime. When someone misses a forecast and comes back asking for two or three times the wafers, it often takes six or eight months to sift through. It is not only that Wall Street missed what Nvidia would sell. Nvidia missed it too. You can ask TSMC for wafers, but many of them are already allocated to AMD, Qualcomm, and other giants.

Will forecasts get accurate after this discontinuity? No. We are going to continue to get it wrong for a while, and he thinks the miss will keep being to the underside. The market is accelerating; two or three years from now it will be many times bigger than anybody currently thinks. The other ugly dynamic is that if you order too much, you still have to take it. Ten million dollars of wafers are yours. That is what happened to Nvidia in crypto, and to Broadcom. Arista, he notes, is at 52-week lead times on switches.

As models scale, inference cost starts to dominate, and Feldman treats the size question as an optimization problem rather than a law of nature. Bigger models are harder to work with and harder to retrain. When GPT-4 came out it had no insight beyond 2021, a lag he attributes to how large it was. Accuracy trades off against size; size trades off against serving cost. You might want a larger model for radiology files and a smaller one for a customer-service bot. Think about what the model delivers, the cost of being wrong, and the cost of serving, and treat it as a business decision. After the first wave of "I can make a bigger model," people who run businesses will sit at three, six, and 13 billion parameters, he predicts — pretty darn good without breaking the bank on free inference.

Fine-tuning, including the then-new OpenAI and Scale partnership, will matter. Human feedback makes models vastly better, which does not surprise him: that is how you learn to dance, play soccer, do gymnastics, ballet — someone saying no, do it again. Nobody yet knows how far fine-tuning will go, or who benefits: training from scratch, continuous training, or taking a model off the shelf and adding unique data. Large labs sometimes argue that very few people will train. Feldman points at enterprises sitting on extraordinary datasets — Reuters, Bloomberg, GlaxoSmithKline, AbbVie — years of research that AI is now good enough to use. Data as an asset, he says, is the new gold, under-talked-about today and very much top of mind going forward.