Anastasios Angelopoulos on Arena, Open Models and AI Evaluation

Anastasios Angelopoulos, co-founder and CEO of Arena, describes it as a platform for measuring how AI performs when real people use it. Rather than relying only on static benchmarks, Arena gathers comparisons and feedback about tasks, preferences and errors, then helps labs and users interpret the results.

Harry Stebbings asks whether the number of models makes them a commodity. Angelopoulos says closed models remain concentrated among a small group of companies, while rapidly improving open models introduce more competition. His assessment concerns the market at the time of the interview.

He says Kimi K3 surpassed leading American closed models on a subset of Arena tasks, including some front-end coding. He considers that a notable result, but it is a task-specific claim based on Arena's measurements, not a finding that one model is best at everything.

Angelopoulos argues that this result challenges the idea that Chinese labs can only keep pace by distilling American models. He explicitly says it does not prove that distillation was absent; in his view, other work must also contribute to performance.

Asked whether Chinese open models are displacing closed providers, he cautions that OpenRouter's traffic is not the whole market. He believes much inference still goes through first-party APIs and proprietary models; that is his estimate, not a comprehensive independently verified market count.

Looking ahead, Angelopoulos expects enterprises to want control over their AI supply chain: a model they can adapt on their own data and run in their own environment. He connects this preference to cost, control, improvement and concern about relying on an outside provider that could become a competitor.

He argues that proprietary data and network effects could remain sources of advantage as software becomes easier to produce. His claim that company-specific intelligence will become widespread is a forecast, not an established outcome for every industry.

Asked whether US companies will rely on Chinese open models, he expects an American open competitor to emerge over time, partly because of the regulatory environment. His prediction of a very large US open-model company is speculative, not an estimate of a current company's value.

Angelopoulos describes possible open-model business models: sharing revenue with inference providers, or using an open model to attract customers for fine-tuning, deployment and AI modernization services. Stebbings notes that closed-model providers also offer deployment help; Angelopoulos sees the combination with an open model as the distinguishing proposition.

He expects businesses to spend heavily on integrating AI into workflows, organizing data and training employees. Both the size of that future market and the ability of open-model providers to capture it remain forecasts.

The conversation turns to Thinking Machines and its Inkling model. Angelopoulos presents two readings of its progress: a shorter development period after a team restructuring versus the longer life of the company. He says Arena's leaderboard at the time placed it first among American open models but behind several Chinese models.

Stebbings shares a Chinese researcher's view that harder work and policy support help explain performance. Angelopoulos says Chinese teams face both advantages and constraints, and notes that access to advanced chips is one possible limitation. Neither account establishes a single cause of the performance gap.

He says export restrictions may slow access to hardware now while also encouraging a domestic alternative over time. The strategic question, in his view, is whether to keep the world using American hardware or restrict access to preserve a near-term advantage; he treats the tradeoff as a real debate.

The speakers consider possible restrictions on model access. Angelopoulos outlines potential security and industrial-policy arguments for restrictions, as well as the cost to companies that would lose access to useful models. His descriptions of the regulatory landscape are his views, not legal guidance.

Asked whether hosting a model locally removes all risks, Angelopoulos describes a hypothetical scenario in which model behavior could still be exploited through an application. He presents a possible attack vector, not evidence that a particular Chinese model contains a backdoor.

He guesses that restrictions on access to Chinese open models may exist in three years, while emphasizing uncertainty and not endorsing that outcome. Stebbings separately speculates about the lobbying influence of large American labs; the interview does not establish a coordinated effort.

Angelopoulos advocates a strong open ecosystem and incentives for American open models. He also acknowledges that chip suppliers may benefit commercially from more model training and fine-tuning, even when he agrees with their support for openness.

Would Arena survive without open models? Angelopoulos says it could still provide evaluations in a market with several closed providers, but believes a single provider would threaten its business and that two would make the outlook difficult. This is his assessment of Arena's own incentives.

He says some enterprises he has spoken with worry both about frontier labs and Chinese open models. As one anecdote, he recalls a prospective customer asking whether Arena could replace a Chinese model in its stack with an American one. The example is not a survey of all enterprises.

Discussing American and Western open-model contenders, he mentions several labs and praises the efficiency of Google's Gemma on an Arena performance-versus-cost view. Stebbings discloses that he is an investor in Poolside, whose model both speakers discuss favorably.

On routing, Angelopoulos sees value but also hype. A useful router, he says, must understand the nature and difficulty of a request, compare models using relevant data and incorporate new releases quickly. He does not claim to know how every competing router works.

He expects outcomes to depend on execution, cost, performance and whether a company makes routing a priority. He says the competition is unresolved rather than declaring a winner.

Angelopoulos predicts that inference will become cheaper over the long run. He speculates that more public financial information could strengthen customers' negotiating position, while Stebbings points out that a differentiated product may retain pricing power even if its costs are known.

Asked which frontier lab might reach public markets first, Angelopoulos offers a prediction and discusses reported readiness. His statements about companies' financial positions and the timing of any listing are not independently verified in the interview and should not be treated as confirmed plans.

The speakers discuss a reported security incident involving OpenAI and Hugging Face. Angelopoulos considers it significant and gives his understanding of the event; the interview does not independently establish its details or attribution.

His proposed response includes stronger access controls and external safeguards. He describes a guardian model that watches agent activity and flags actions that appear unsafe, while recognizing the challenge of matching the capabilities of the system it monitors.

Angelopoulos opposes a central authority approving each model release. He prefers safety incentives, accountability and rules tied to outcomes, including possible penalties for harmful incidents. These are his policy preferences, not evidence that any safeguard can guarantee security.

He expects more cyber incidents and recounts what he says was a fraudulent job applicant who passed interviews but could not be verified as a real person. He says he does not know whether the motive was espionage, financial fraud or something else, and does not attribute it to a particular state.

Arena is considering in-person onboarding and identity verification in response, he says. His account is an anecdote about the company's hiring process, not an independently verified description of the applicant or a measure of how widespread such attempts are.

Hiring researchers in the Bay Area is highly competitive, Angelopoulos says, with experienced specialists commanding substantial compensation. He also sees some researchers leaving large labs for smaller organizations where they believe they can have more impact.

Stebbings asks what will separate successful new model labs from failures. Angelopoulos estimates that many will have little standalone value or be acquired for their teams. That is a speculative forecast, not a count of businesses already failing.

He argues that a strong research team alone is not enough: labs need a strategy, sustainable business model and revenue growth to support subsequent financing. A company with a high valuation and little revenue must eventually justify that valuation to investors and employees.

He illustrates the financing problem with a hypothetical $10 billion valuation and a future $100 billion outcome, using revenue multiples to estimate the growth required. The numbers are an example of his reasoning, not a valuation or revenue forecast for a named lab.

Stebbings notes that secondary sales can give employees liquidity before an exit and points to reported revenue at established companies. Angelopoulos distinguishes those businesses from startups valued highly before they have revenue or a clear plan.

Both discuss the possibility that investors price in an acquisition of a talented team as a downside case. Angelopoulos says this may help explain early funding but makes the next round harder if the business cannot demonstrate progress; an acquisition is not assured.

The discussion shifts to the market for training data. Stebbings discloses an investment in Mercor and cites revenue figures for several providers. Angelopoulos says these businesses have grown rapidly, while the specific figures are their discussion and are not verified in the interview.

Angelopoulos calls data a complement to model scaling: in his economic analogy, more demand for one good can create demand for another. He expects more models and more companies training their own systems to increase the need for data, but acknowledges the argument rests on continued AI growth.

He estimates that frontier labs spend on data an amount equivalent to roughly 10–20% of their GPU spending and predicts a market of at least $100 billion by 2030, possibly more. Both the spending estimate and future market size are his claims, not independently established figures.

He says collecting, cleaning and creating useful training data is difficult and predicts that major providers could become very valuable. The proposed company valuations are speculative; the practical challenge he identifies is making high-quality data available for a particular training task.

Asked whether dependence on a few large customers undermines data companies, he argues that customer concentration can coexist with valuable businesses. He also expects providers to expand into enterprises seeking their own models and data, although that diversification is a prediction.

Angelopoulos says agent evaluation is Arena's top priority. He reports more than 30 million monthly visitors and describes many users as knowledge workers performing real tasks; these are company-reported figures and characterizations rather than independently audited measurements.

He says feedback from those interactions helps Arena create evaluations and give model labs information for improvements. He wants to offer similar evaluation capabilities to businesses deploying agents and deciding which models fit their work.

He separates evaluation into performance, cost and latency. Cost and speed are easier to quantify, he argues, while performance depends on the business and the use case. Arena's proposed approach uses information from agent activity to help companies evaluate their own workflows.

Asked about revenue, Angelopoulos says Arena has passed a $100 million annualized revenue run rate, calculated as second-quarter revenue multiplied by four. That is a company-reported run rate, not a statement of $100 million in recognized annual revenue.

He says Arena is not yet free-cash-flow positive because it is investing in growth and product development. He describes the business fundamentals and margins favorably, without giving an audited margin figure.

On infrastructure business models, Angelopoulos says profit depends on both margin and volume. He urges investors to consider the lasting value of a service that resells tokens or compute, and how its margin structure might evolve as it scales.

Stebbings asks whether frontier labs will move aggressively into applications and compete with companies that rely on their models. Angelopoulos says he has heard of customers choosing model labs directly and considers this a risk for application companies, including legal AI providers. These are his accounts and predictions, not evidence of a particular company's demise.

When Stebbings notes that some customers fear frontier labs while others embrace them, Angelopoulos says both behaviors exist. Stebbings argues that distribution and deployment differ greatly between products for designers and tools sold into law firms.

Angelopoulos adds that businesses with operational complexity or network effects may face different competitive dynamics from a simple software application. His examples are strategic judgments about where model labs may or may not compete, not guarantees.

On Salesforce, he says the company has an AI strategy and believes predictions of a broad software collapse can underestimate established networks and data. Stebbings adds that the degree to which a product is embedded in enterprise workflows matters. Both assessments are forward-looking.

In the quick-fire round, Angelopoulos says open models and Anthropic have advanced faster than he expected. He has also learned that managing people, setting strategy and planning ahead are central to leading Arena, alongside technical work.

He still sees university as a place to develop first-principles thinking, intellectual habits and relationships. Reflecting on Arena, he says he wishes he had listened more to his co-founder and spent less time on early experiments that distracted from what worked.

The conversation returns to application competition. Stebbings argues that a focused legal product can benefit when its category is a lower priority for a frontier lab, and Angelopoulos agrees that focus matters. Their exchange refines, rather than eliminates, the competitive risk discussed earlier.

Asked which company might first reach a $10 trillion valuation, Angelopoulos picks Nvidia as a likely leader and predicts substantial upside from enterprise AI adoption. This is a speculative investment view, not a guaranteed market result.

He also worries that debt financing for compute could become vulnerable if open models reduce frontier providers' enterprise revenue. Stebbings raises the risk of reliance on a small number of companies meeting targets; both describe possible scenarios rather than established insolvency.

Angelopoulos says some consolidation could reduce industry hype and predicts Arena will succeed in its category. Stebbings suggests physical data-center infrastructure may receive less attention than chips and memory; this is their opinion about market attention.

Asked about an underrated lab, Angelopoulos mentions Black Forest Labs while saying many new labs may be overrated. He closes by expressing hope for AI in medicine and notes that biological feedback and data collection remain difficult. These are research hopes, not evidence of a cure or an established treatment.

The speakers discuss the challenge of building data infrastructure for biology and thank one another for the conversation. Angelopoulos invites Stebbings to visit Arena's office.