Lin Qiao on Specialized AI, Open Models and the Economics of Inference

Fireworks AI co-founder and CEO Lin Qiao joins Harry Stebbings to discuss specialized models, inference economics and how founders scale. Stebbings says he invested $10 million in Fireworks; Qiao's arguments for model customization and infrastructure reflect the business her company sells.

Qiao recalls that an investor initially questioned whether a big-tech executive could successfully found a startup, though he honored their agreement. She started Fireworks at 48 after years building data and distributed systems, including work at LinkedIn and Meta.

She had considered starting a company in 2015, with a product plan and potential teammates, but decided she needed to develop her people-management skills. She joined Facebook intending to learn for one or two years and stayed seven.

Stebbings asks why inference and specialization between chips and model providers are valuable. Qiao argues that much useful business data is private and excluded from general model training, so companies can derive distinctive value by applying AI to their own data.

Qiao offers her own definition of AGI as one model that solves every problem best, then questions whether one system should impose a single set of tastes and judgments. That is her framing, not a universally accepted definition of AGI or a characterization of any lab's stated goals.

Different regions, businesses and users have different values and needs, she says. Qiao sees creativity and choice as reasons to preserve diversity in AI systems; her concern is that dependence on one provider could narrow those choices.

Qiao recalls a conversation with Nvidia's Jensen Huang about companies being built around distinctive beliefs and products. She argues that an organization's particular knowledge is embedded in its products, user interactions and data, and may not be readily captured by an outside provider.

She compares frontier models to power lines: foundational infrastructure supplies intelligence on which many specialized products can be built. Stebbings asks whether that infrastructure will remain a good business if open models improve and become cheaper.

At Fireworks' founding, the team debated training its own model versus building on open models. Qiao says the latter was a substantial bet at the time, informed by the team's experience with PyTorch and a belief that access to model weights can give users more control.

Qiao says both open and closed models have improved enough to solve many practical problems. At Fireworks, she describes using open models for recruiting, internal finance and coding, while remaining conscious of cost. These are examples from her company, not a general performance benchmark.

She argues that open models have become easier to tune with a company's own data and evaluations. For a particular task, she says, a customized model can outperform a general-purpose model; performance depends on the task, data, evaluation and models being compared.

Stebbings asks whether frontier labs are overvalued if open models handle most enterprise tasks. Qiao shifts to a distinction between product-market fit and durable economics: a company can have customers who value its product yet face costs that make scaling unsustainable.

She says large digital businesses may face the same challenge when an AI feature reaches an enormous user base. Her proposed alternative is to control and customize open-weight models. Stebbings notes that frontier providers could also cut prices, a competitive possibility Qiao acknowledges.

Qiao argues that acquiring released open weights can have no model license acquisition cost in some cases, while providers of closed models must recover research costs. Running, adapting and serving an open model still require resources; she emphasizes the control to tune and deploy it for a specific workload.

Fireworks says it tunes and deploys models for individual use cases, optimizing quality, speed and cost. Qiao says even a 5% cost reduction can matter at very large scale and cites reductions of five to ten times in past cases, without presenting comparative details in the interview.

Stebbings raises national-security concerns about Chinese open models. Qiao says the issue is debated and recommends that businesses apply their own guardrails to both open and closed models because any provider's judgments may differ from a customer's. Guardrails and tuning are her proposed measures, not proof that security risks are resolved.

Qiao predicts millions of specialized models rather than a world dominated by a few general ones. Asked about reports that China might restrict access to its open models, she expects short-term disruption but believes a diverse open ecosystem and US capabilities could eventually provide alternatives.

Stebbings discloses an investment in Legora as he asks whether legal AI companies such as Legora and Harvey need their own models. Qiao says AI-assisted coding can shorten development, but Stebbings counters that enterprise sales, relationships and deployment remain difficult.

Qiao agrees legal applications are demanding because mistakes and hallucinations can have serious consequences. She says both companies may have valuable knowledge and data about legal workflows, while acknowledging that her own understanding of the legal field is shallow.

She describes an assistant's harness as the layer that orchestrates tools and models. In her view, improving the model and the harness together can help an application use its domain knowledge more effectively; she cites coding companies' model tuning as an example.

Asked whether rapid model advances will slow, Qiao distinguishes major changes in general model capability from specialization built on top. She expects general advances to continue in steps and specialization to accelerate across many applications; this is her forecast, not a measured timeline.

Stebbings asks whether governments may own part of foundational AI infrastructure. Qiao says she does not know, but argues against a future in which one company controls intelligence. She compares general models to shared infrastructure while favoring many specialized approaches.

In a multi-model world, Qiao imagines routing difficult decisions to highly capable models and narrower subtasks to smaller or customized models. She expects automated routing and tuning to improve together as products and their users change.

Stebbings asks whether an independent routing layer remains valuable if customers automate or build it themselves. Qiao says they may not need a separate provider in that case, while considering routing and automation an area for innovation.

The host asks about Fireworks' close work with Cursor, including embedded technical support. Qiao says early adopters often want detailed control, while later users need accessible systems with less operational complexity. Fireworks aims ultimately to serve the latter but learns from demanding early partners.

Qiao describes Cursor and Fireworks as capital-conscious startups seeking to advance research without building one enormous, expensive interconnected cluster. Their work separated the model trainer, which updates weights, from reinforcement-learning rollouts, where model versions interact with environments and receive feedback.

According to Qiao, the system uses GPUs across five or six data-center regions. The engineering challenge is synchronizing updated model weights quickly enough that feedback does not become too stale. She says their distributed approach supports Cursor's model launches; this account is from a commercial partner.

Stebbings asks about dependence on Cursor following a SpaceX acquisition. Qiao says the industry has been concentrated around a small group of successful applications, but describes Fireworks' customer base as increasingly diversified; she does not quantify the change or guarantee retention.

She calls the prior year the year of coding and the current one the year of co-work, with applications spanning research, legal, finance, customer support, recruiting and other domains. She also sees consumer-facing businesses exploring generative AI for recommendation and related products.

Qiao says customers often use multiple inference vendors to manage uncertainty. Fireworks positions itself around customized intelligence rather than commodity inference; she says the platform processes more than 40 trillion tokens per day, mostly from customized models. That usage figure is her company claim.

Asked about the following year, Qiao says token volume could grow 20 to 100 times. She sees the market as early in an adoption curve, but the range is a speculative forecast, not a committed target or established demand.

Stebbings suggests such growth would imply a need for much more compute investment. Qiao agrees about potential demand while identifying constraints in energy, chips and physical supply chains, including small components that can hold up server production.

Does a company need to own every layer from chips to applications? Qiao says Fireworks prioritizes agility and focus, wants to run on many chip types and prefers to use partners' strengths. As a company grows, she says, building more infrastructure itself may become economical.

She recalls Cursor choosing Fireworks early so it could concentrate on product and research rather than platform infrastructure. Qiao gives a very large estimate of Cursor's subsequent growth, presented as her recollection rather than independently verified company data.

Asked why Nvidia does not build the same platform layer, Qiao offers her interpretation of Huang's strategy: Nvidia focuses on its strengths and supports US open-model supply where it sees a gap. She does not speak for Nvidia, and this is not a statement of its official strategy.

Stebbings asks whether AI spending as a share of developer salaries will rise as tools improve or fall as they become cheaper. Qiao predicts a substantial decline in token costs as supply expands and competition increases, followed by much greater usage.

She cautions that tokens are not interchangeable: a model priced at half the cost per token may use twice as many tokens to complete a task. She therefore proposes evaluating the cost and quality of a completed task, not the token price alone.

Qiao identifies three potential sources of savings: fewer tokens per task through model improvements and tuning, more efficient inference, and eventually less constrained GPU and memory supply. She does not expect the supply situation necessarily to change within a year to 18 months.

Over roughly three years, Qiao says she can imagine a tenfold reduction in cost driving a hundredfold increase in usage. The two figures are her contingent forecast, not a demonstrated relationship or a promise that prices and demand will follow that path.

Stebbings says he found a competitor cheaper and asks whether Fireworks charges more for quality. Qiao says comparisons may not be like for like because much of Fireworks' traffic uses customized models, and the company prioritizes task-specific model quality as well as serving performance.

Qiao describes a technical goal of preserving numerical equivalence between training and inference, which she says helps customers retain the value of training. She presents Fireworks' approach as differentiated from a uniform off-the-shelf deployment; the interview does not independently verify the claim.

Fireworks has an applied machine-learning engineering team that helps customers deploy customized models and build automation, Qiao says. Stebbings asks whether infrastructure businesses face permanently lower margins than traditional software companies.

Qiao says current margins at Fireworks reflect hypergrowth, not necessarily a permanent industry norm. In her view, imposing aggressive margin targets too early can constrain experiments, geographic expansion and product development; she expects more optimization after the business understands which systems to scale.

She says Fireworks will not move into the application layer. Building data centers remains a possibility depending on timing, but she does not commit the company to doing so. She also says the company can improve margins without necessarily entering another layer.

Qiao explains that uniform hardware fleets are easier to operate, while mixing different accelerators might better match prefill and generation workloads. She describes this as an engineering trade-off requiring specialized system design, not a universal performance result.

Data-center construction and operations involve power, networking, cooling and replacement of failed parts, she says. Qiao stresses that operating such facilities requires expertise and cannot be mastered overnight.

Stebbings asserts that Chinese construction faces fewer barriers and can move faster. Qiao shares personal observations of fast construction in China and slower local work near her home, while noting supply constraints and electrician shortages. Their anecdotes do not establish a comprehensive comparison of national rules or construction speeds.

On hardware economics, Qiao says fast model and chip releases complicate depreciation and build-versus-buy decisions. Equipment may physically last years while customers prefer newer hardware for newer models; she treats the future value of older systems as uncertain.

Qiao says Fireworks wants both growth and healthy gross margins, improving differentiation continuously without sacrificing growth to an early margin target. She argues that maximizing the margin by stopping growth would be a poor outcome for the company.

Asked if a low-margin strategy could create a winner-take-all market, Qiao says leadership is a long-term question and expects sectors to settle around a few strong providers. She sees customers moving toward customized models as their deployments mature and optimization becomes more valuable.

Stebbings asks about sovereign models and the risk of relying on a foreign provider. Qiao says national ownership is possible and compares access to AI infrastructure with access to power; she favors control at both country and company level. The host's account of a temporary restriction is presented as his premise, not independently established here.

Why consider data centers but not chips? Qiao says chip design is extremely difficult and becomes more sensible when a large workload is stable enough to justify specialized hardware. She cites Meta's years of hardware work as context, not evidence that every lab's reported chip plans will succeed.

She considers current AI workloads too dynamic to know which patterns will remain durable. Some accelerators designed before the generative-AI wave happen to fit current workloads well, she says, while others do not; the outcome depends on the workload and hardware design.

Qiao identifies systems for very large models as an underappreciated engineering challenge. She wants coordinated design across the model, customization and serving layer, and collections of chips, and suggests there is room for innovation in systems for extremely large parameter counts.

Stebbings cites a recently announced $800 million annual recurring revenue figure and asks where it will be by year-end. Qiao says Fireworks believes it can at least double it. The figure is the host's statement and the increase is the CEO's forecast, not a verified future result.

Qiao attributes unusually rapid company growth in part to broad demand for tools that enable more people to create. She then discusses hiring former Salesforce executive George Hu: when Fireworks was around 50 people she felt the company was too small, but he helped with executive hiring before joining more closely as it grew.

She values Hu's experience alongside his curiosity and willingness to learn in a fast-moving AI market. Qiao says his go-to-market team uses AI agents to improve productivity while building capacity to meet demand, a description of Fireworks' own operations.

In the quick-fire round, Qiao says she has become more comfortable growing the team quickly after initially worrying about losing agility. She says Fireworks looks for people who take end-to-end ownership, communicate and carry a problem through with colleagues.

Qiao admires Jensen Huang's close attention to detail. She defines leadership as making good judgments with sufficient context, arguing that in a fast market long chains of communication can lose information. Her own approach is to stay close enough to the work to understand what is happening.

She wishes Fireworks had invested earlier in marketing. The team initially assumed a strong product would speak for itself; she now sees marketing as a way to educate customers and explain value, not merely promotion.

Qiao thinks measuring AI return on investment deserves more attention. As deployment expands, she expects companies to focus less on maximizing token consumption and more on attribution, costs and business outcomes; that shift is her prediction.

She says Fireworks has spent relatively little time pursuing traditional enterprises but names customers including Geico, Capital One and Mercury Insurance. She sees that segment as a large opportunity, though the interview does not provide contract details.

For the rest of the year, Qiao wants to prove that Fireworks can scale its team without losing speed. Looking three years ahead, she predicts that companies will increasingly own and customize their intelligence, while deciding which components to build and which to obtain elsewhere.