Li Zhifei on Large-Model Startups and Rethinking His Approach

Zhang Xiaojun interviews Mobvoi founder Li Zhifei about why his views on large-model startups changed in 2023. His judgments and predictions reflect the technology and market context of that time.

Li recalls shifting from computer networking to language and speech research during his PhD at Johns Hopkins. He was drawn to uncertainty in language compared with deterministic systems problems.

Speech recognition must handle noise, speaking speed and accents to convert sound into text. He found it compelling that machines could learn these mappings from data.

He became interested in language understanding and vision, imagining machines handling richer information. This is his recollection of motivation, not evidence of capabilities at the time.

He recalls doctoral study and industrial research, later working at Google before founding a company in China. The settings exposed him to academic, engineering and product questions.

Discussing the evolution of natural-language processing, he contrasts earlier statistical methods with later deep learning: models, data and distributed training all changed.

He says earlier systems were often built around specific tasks and relied more on hand-designed relationships, while deep learning offered ways to learn more complex representations.

As data and compute expanded, tasks such as machine translation improved substantially. The amount depends on tasks and evaluation, not one number for all systems.

ChatGPT drew his attention to a relatively general model handling multiple tasks. He contrasts that with building a separate system for each task.

He recalls discussion of general capability around GPT-3 and says later model performance led the industry to reconsider the research path.

Li explains self-supervised learning: for example, predicting the next word uses the text itself as a training target rather than requiring a human task label for every example.

He contrasts this with parallel translation labels or audio transcripts for speech recognition. Self-supervised training still involves data collection, quality and later adaptation.

He recalls traveling to the United States after the Lunar New Year to speak with researchers and practitioners about the technology, OpenAI’s choices and the difficulty of catching up.

He sees sustained focus on general models as one factor in OpenAI’s path. His account draws on external conversations and observation, not a complete internal record.

He discusses potential barriers involving compute procurement, model architecture, data cleaning and experiment cycles. Specific advantages and costs change with technology and markets.

Large-scale training requires repeated experiments, analysis and revision. A team’s ability to run them reliably depends on people, infrastructure and data processes.

On Google’s organizational structure, he offers an observation that incentives, processes and existing businesses can affect how research becomes products. This is his analysis.

He discusses Microsoft’s collaboration with OpenAI and points to enterprise customers, software and cloud services as application settings. His explanation of motives is speculative.

He recalls offering a rough funding estimate when discussing startup barriers with Wang Huiwen, broken into compute, staffing and data. It was not an actual budget or universal standard.

The host asks why his view changed over roughly a month. Li says learning more about training difficulty and competition led him to reconsider which goals to pursue.

He distinguishes pursuing a frontier general model from building applications around available technology. The routes require different resources and organizational capabilities.

He says Mobvoi had experience in speech and generative applications and hoped to build models and products on that foundation; this does not guarantee later business results.

They discuss model limits and service reliability. Completing a demonstration and reliably serving many users remain separate engineering challenges.

The host asks whether a domestic organization will necessarily pursue the frontier of general models. Li says the outcome is uncertain and the industry may develop different divisions of labor.

He discusses potential applications and social effects of large models, saying language interaction opens product possibilities while creating questions about errors, cost and boundaries.

He expects founders to enter the field rapidly, but the number and competitive structure are predictions from that time, not a substitute for observing later markets.

He compares pressures on an earlier generation of AI companies with those facing new model startups: established firms must adapt, while newcomers face resources, hiring and product challenges.

Asked whether he fears missing the opportunity, Li acknowledges anxiety amid rapid change and many competitors, and the need to judge where he can add value.

He says competition can drive learning, but following a trend does not automatically identify a route suited to a team. Judgment must consider capabilities, resources and real needs.

He recalls that large companies’ attitudes toward AI in 2012 differed from later years. Changes in enthusiasm affect funding, hiring and competition.

An earlier vision of an ubiquitous personal assistant involved home, car and mobile settings. The idea and its feasibility in hardware, experience and business are separate questions.

He reflects on difficulties in earlier approaches and wants to retain the team’s speech and interaction experience while recognizing that new models changed the technical environment.

He says experience includes understanding why earlier methods were limited and which work a team should pursue or avoid investing in blindly.

They discuss the pressure and emotional changes of founding a company. Li says sustained operation means facing uncertainty, not just the excitement of launching a product.

He describes continuing to learn from papers, products and industry information, focusing more attention on technological change and its implications for the team.

They discuss researchers and founders he admires. He values people who understand technical details and limitations alongside broader questions.

He says the relationship among language, internet data and intelligence cannot be answered by one paper; specific algorithmic research and broader thought can inform each other.

He reflects on how his personality changed through entrepreneurship, saying sustained difficulty made him humbler about operating a company and his own judgment.

He discusses the pressures of teams, responsibility and decisions. Common success narratives can omit long periods of adjustment and failure.

He admires people oriented toward the future and broader social progress. This reflects personal values, not an objective ranking of founders.

Asked for advice on pursuing general models, Li urges founders to assess investment, team and differentiation rather than expand without resources and a plan.

He says smaller teams can seek advantages in efficiency, cost or particular use cases, but such advantages require validation through products and outcomes.

The conversation closes on founders continuing to read and learn about technology. The interview records observations from 2023, not today’s market or model capabilities.