AI Safety, Personal Agents and the Venture Market’s Uneven Exits

Harry Stebbings opens the roundtable with Jason Lemkin and Rory O’Driscoll by asking about calls to pace frontier AI development and possible external oversight. The three debate how companies and governments should handle risks whose likelihood and timing are uncertain.

O’Driscoll says the discussion should separate several questions: misuse, economic disruption and loss of control. He criticizes some extreme public claims while distinguishing them from the particular concerns he attributes to Anthropic CEO Dario Amodei. His assessment does not establish how likely any risk is.

O’Driscoll criticizes the idea of assigning precise catastrophic probabilities without an agreed basis, while also saying that different researchers make different claims. The panel does not examine the underlying evidence in depth, so its skepticism is not a measurement of the risk.

He says cybersecurity risks warrant attention, while expressing skepticism about predictions of widespread unemployment. He is less certain how to assess a possible loss of control. In his view, voluntary monitoring, government regulation and international coordination each raise different practical questions.

O’Driscoll argues that if governments decide oversight is needed, public regulators rather than the companies alone would determine the rules. He worries about innovation and implementation and says he understands objections both from people who dispute the need and from those who question industry-designed oversight.

Lemkin offers a different interpretation, speculating that a public discussion of risks could also prepare a company for questions in a possible future IPO. He describes this as his personal theory about corporate communication; the panel does not establish the motives behind any statement or the timing of an offering.

The group distinguishes statements made by individual researchers from the positions of a company’s CEO. O’Driscoll calls for public scrutiny of serious claims, while Lemkin predicts extensive political attention. Neither the proposed hearings nor their possible consequences are established events in the conversation.

Lemkin says non-technical acquaintances have asked him about AI safety and expects the topic to become more prominent in public debate. O’Driscoll agrees that political and regulatory response could become an additional business risk. These are forecasts about public reaction, not evidence that a particular action will occur.

The panel discusses the responsibility of a company building a potentially risky product. Lemkin argues that executives must address material hazards rather than leave responsibility entirely to outsiders. O’Driscoll agrees that company leaders have obligations, while acknowledging that the exact nature and extent of future risks are disputed.

Stebbings asks whether systems that improve themselves could develop in ways their creators do not expect or could be misused. O’Driscoll regards loss of control as a serious uncertainty and argues that builders should not continue unchecked if they believe they cannot manage a material risk.

O’Driscoll says the possibility of losing control is harder for him to dismiss than some other concerns. His preferred response places primary responsibility on developers when a material hazard is identified, while the panel leaves open how independent oversight might assess that judgment.

Stebbings and O’Driscoll disagree about how much the development of open models in China depends on the work of US labs. O’Driscoll says he cannot assess specific technical claims and cautions against crediting any one group with all progress. The exchange is opinion, not a finding about a particular organization.

The panel returns to how public statements affect trust. Its members note that different people have made different claims, and that attributing a researcher’s view directly to a company leader can be misleading. They expect public debate about employment, security and governance to continue.

Stebbings says prior public warnings about employment can influence how later statements are received. O’Driscoll responds that communications about risk can affect public trust. Their discussion concerns perception and does not determine what any individual privately believes.

Lemkin raises the general dual-use problem: capabilities that help people can also be misused. He argues for safeguards, particularly where powerful systems are widely available. The panel does not establish that every capability automatically has a corresponding harmful use or that any particular safeguard is sufficient.

Asked about infrastructure markets, O’Driscoll describes his interpretation of a short-term shift between semiconductor and cybersecurity stocks. He treats this as a snapshot, not proof that a particular safety announcement caused a price move or that market prices measure AI risk.

Lemkin argues that the prospect of large financial returns can encourage risk-taking and make it harder to slow a fast-moving industry. O’Driscoll agrees that economic incentives and public concern can coexist. Their exchange does not quantify the prevalence of harmful motives among founders.

O’Driscoll observes that people may express concern about technology while continuing to invest in it. The panel suggests that financial incentives and a belief in beneficial uses can pull in different directions, without resolving how any individual company should balance them.

The group turns to Meta’s Muse and the personal-assistant category. Lemkin says his early experience was positive: in his tests, the product completed several everyday tasks and helped with a website. Stebbings says he could not access the service directly from the UK at the time of recording.

Lemkin says he tried Muse on tasks such as organizing information and changing a website, and was impressed that it completed them. The examples are his initial experience, not an independent test of reliability, privacy or performance across users.

Lemkin argues that serving an agent can involve meaningful compute and hosting costs. He believes Meta benefits from existing infrastructure and its own models, though his estimates of per-user costs are illustrative rather than audited product economics. He sees infrastructure as a possible advantage over smaller competitors.

Despite praising the implementation, Lemkin asks what will make users return frequently. He is not sure that occasional reservations or messages constitute a defining use case. Stebbings counters that many modest improvements across travel, shopping and calendar tasks may together make an assistant valuable.

Stebbings asks whether an assistant could become a way to discover and purchase products, similar to a super-app. Lemkin says he wants to see a sustained reason to use it frequently; Stebbings argues that convenience across several activities could be enough.

Stebbings says Instinct has helped him with several tasks but reports that response latency can be frustrating. His experience is one user’s account, not a comparative benchmark. O’Driscoll says the category may work even without a single dramatic use case if the combined interaction is convenient.

O’Driscoll compares the ambition with earlier super-app and messaging experiments. He suggests better models could make mobile interactions more useful than older chat interfaces, while asking whether the winners will be standalone companies or services embedded in large existing platforms.

Stebbings describes a reported Instinct financing of $1 billion at a $10 billion valuation and opens a mock investment committee. Lemkin initially plays the role of an advocate, arguing that a focused team might serve users across services and compete with a platform whose attention is divided.

In the hypothetical pitch, Lemkin raises questions about interoperability and whether a large company will sustain investment in a product with little immediate revenue. Those are competitive assumptions, not verified limits on Meta’s product or a prediction that a named employee will leave.

After making the assigned case, Lemkin says he would not personally invest in that round. He worries that Meta can combine its own models, compute, distribution and capital with a strong interest in building the same application. He acknowledges that his judgment could be wrong.

He illustrates the economics with a hypothetical per-user infrastructure expense: a large user base can produce a substantial annual bill before the business has found sustainable monetization. Venture capital may subsidize that cost for a time, but the relationship among usage, pricing and renewal remains uncertain.

O’Driscoll agrees that a personal assistant appears strategically central to Meta in a way that some earlier enterprise products may not have been. He also notes a possible alternative outcome for a successful startup: a large buyer might want its team, users or technology. No acquisition is presented as agreed or guaranteed.

Stebbings says a founder with exceptional talent might attract a major strategic buyer and mentions a hypothetical very large sale. Lemkin cautions against building an investment case that requires one particular acquisition outcome because buyers, priorities and timing are unpredictable.

Lemkin says a large strategic acquisition can depend on a buyer’s changing priorities and individual decision-makers. A plausible buyer today may not be available when a startup needs an exit, making the outcome difficult to underwrite.

O’Driscoll considers whether a portfolio could include a small number of high-variance bets with acquisition outcomes, even though that is not his own preferred approach. He stresses that expected value and risk depend on price, probability and the rest of the portfolio.

O’Driscoll emphasizes that a portfolio approach cannot remove uncertainty from an individual deal. Even if some high-risk investments have positive expected value, the investor still needs enough capital and diversification to absorb failures.

The investors compare the risk-return profile of an early round with later rounds at much higher prices. O’Driscoll says a potential outcome that creates a large multiple from a low entry price produces a smaller return at a higher price, even if the eventual sale value is unchanged.

Stebbings reports strong interest after a social-media post about Instinct, including many requests for invitations. The panel takes the anecdote as a sign of curiosity, while the broader questions of sustained usage, retention and monetization remain open.

Lemkin describes an investor framework centered on very large technology exits. He says that if the number of such outcomes expands, higher entry prices may be easier to justify, but acknowledges uncertainty about how the overall market arithmetic works. This is a scenario, not a guarantee of future exits.

The conversation shifts to Miro, a collaboration and whiteboard company. Stebbings cites its earlier reported $17.5 billion private valuation and a reported sale to Bending Spoons for roughly $1.35 billion. He says outcomes differed across investors, founders and employees; the panel does not independently verify the transaction’s full terms.

O’Driscoll says a stale valuation can become difficult to sustain when market conditions and growth change. He views a sale as one way to resolve that gap and says late-stage investors may have contractual protections, though the treatment of any particular investor depends on the actual deal documents.

O’Driscoll suggests that late-stage investors can sometimes limit losses through contractual preferences and describes a one-times recovery as a favorable result on a weak deal. Whether such a recovery occurs depends on the contract, the available proceeds and the rights of other investors.

The speakers discuss a buyer strategy of cutting costs and raising prices to improve cash flow. O’Driscoll suggests a single owner may find it easier than a venture-backed board to make a sharp operating change. He does not claim to know the precise plan or future performance of this particular acquisition.

He suggests that different ownership structures can produce different operating choices: a venture-backed company and an acquisition-focused owner may have different goals for growth, cost and cash flow. Stebbings questions the effect those choices can have on customers.

Stebbings raises concern that price increases and cost cuts can cause substantial customer churn. O’Driscoll argues that a buyer may accept losing price-sensitive customers if the remaining users value the product enough to pay more. Their examples are illustrative; actual results depend on customer behavior and execution.

Lemkin worries that there may be few attractive exits for mature private software companies with slowing growth and high earlier valuations. He compares the market to a shrinking number of available seats, while conceding that more deals may still occur. This is his market view, not a count of confirmed buyers.

Lemkin cites Miro as an example of a business that, in his account, still had substantial recurring revenue and positive cash flow yet sold at a much lower valuation than its earlier round. He fears weaker companies may have fewer options. The metrics are cited in the discussion, not independently verified.

The panel briefly discusses the legal case involving Sam Bankman-Fried. Lemkin speculates about possible appellate arguments and outcomes, while O’Driscoll says the legal process should determine the relevant facts. Their discussion does not establish that an appeal will be heard, a sentence changed or anyone released.

O’Driscoll argues more generally that the later performance of an investment does not by itself resolve whether funds were used appropriately when an earlier decision was made. He acknowledges that legal terms and facts matter. This is a participant’s view, not a ruling or legal advice.

The group then discusses a reported leadership and board dispute involving Matt Mullenweg and Automattic, the company associated with WordPress. The speakers describe a change and reversal in leadership, but do not establish the governing documents or all events behind the reported dispute.

O’Driscoll criticizes the company’s handling of a conflict involving the WordPress ecosystem and WP Engine. He sees tension between operating a commercial company and stewarding an open-source project. His account reflects his own interpretation; the conversation does not adjudicate the parties’ legal or factual disagreements.

He argues that the broader challenge for any established software platform is how to remain useful as AI changes the way websites and applications are built. In his view, governance conflict can distract from that strategic work. Other market outcomes for WordPress and Automattic remain possible.

O’Driscoll offers a hypothetical explanation based on board and shareholder powers for how a leadership dispute might unfold. He explicitly frames it as a guess, not knowledge of Automattic’s bylaws or internal votes. The interview does not verify his reconstruction.

Lemkin considers whether some software businesses might prefer remaining independent and profitable rather than raising venture capital. He sketches a hypothetical smaller company with different obligations and priorities. The example is not a verified description of Automattic’s finances or Mullenweg’s private motives.

O’Driscoll agrees that founders should understand the obligations that come with venture investment, while separating that issue from the responsibilities involved in leading an open-source community. The panel treats commercial ownership and community stewardship as related but distinct questions.

Lemkin says open-source companies face difficult choices over how much revenue to capture from their surrounding ecosystem. He argues that giving partners room to build can also leave money with other firms. His comments about the WordPress ecosystem are his perspective, not a conclusion that any party acted improperly.

The panel contrasts slower-growth software companies facing difficult exits with AI startups raising at rapidly increasing valuations. O’Driscoll says venture outcomes are unusually uneven: ownership in a fast-growing category may be valued very differently from an older product with stable but modest growth.

Stebbings recalls a conversation with an experienced investor who regarded the market as unusually extreme. O’Driscoll compares the enthusiasm and uncertainty with the dot-com period, while saying the scale of capital and imagination around AI feels larger to him. This is historical perspective, not a forecast of a repeat crash.

Stebbings cites a reported €3 billion Mistral financing and says the company has faced criticism despite its growth. The panel does not independently verify the funding terms or the future revenue figure mentioned in the episode.

O’Driscoll argues that demand for European AI capability may be driven partly by sovereignty: governments and customers may want alternatives they can access and influence locally. He compares this strategic rationale with earlier European industrial efforts. It is his interpretation, not proof of every investor’s motive.

The discussion includes recollections about access to US models and political decisions. O’Driscoll speaks from memory, and the conversation does not establish the precise policy, timing or scope of any restriction. The broader question is how dependence on foreign providers could shape customer preferences.

Lemkin notes that a strategic investor can have objectives beyond an ordinary financial return. O’Driscoll agrees that the cash contributed to a financing matters while the valuation may reflect strategic considerations as well as financial comparisons. They do not claim to know the confidential negotiation behind the round.

Stebbings asks about Adobe’s reported leadership change. Lemkin is critical of the succession and skeptical that it signals a major strategic shift. O’Driscoll also questions whether AI-specific metrics alone demonstrate an acceleration in overall business growth. Their judgments and financial descriptions are opinions at recording time.

Lemkin argues that Adobe’s scale and existing cash generation give it more room than a smaller software company to weather a transition. He nevertheless wants evidence that new products can sustain growth. O’Driscoll distinguishes customer adoption of a new feature from a measurable increase in the whole business.

Asked to predict Adobe’s future market value, the speakers offer rough guesses and discuss cash-flow multiples. They also acknowledge the possibility that a successful AI product could change the trajectory. The figures are speculative and should not be treated as an investment recommendation.

Stebbings asks whether they would prefer Canva or Adobe. Lemkin stresses the difficulty of slowing growth; O’Driscoll clarifies that, in the figures he cites, Canva is still growing but at a lower rate. The distinction matters: deceleration does not mean revenue has stopped growing.

O’Driscoll says investors often change their valuation framework as a company matures. A fast-growing company may be assessed on revenue and expansion, while a slower one may be assessed more heavily on earnings or cash flow. A shift between those frameworks can pressure valuation even when the business remains substantial.

He recalls a public software company navigating a similar transition and says it can take years for cash flow to catch up with the earlier growth valuation. The problem can be especially difficult for a private company seeking liquidity during the adjustment, though public companies face their own investor-base changes.

The panel hopes that a founder-led company can reaccelerate, but recognizes that neither renewed growth nor a particular exit price is assured. O’Driscoll says a profitable company can remain viable even when its growth and valuation multiple fall; the outcome depends on its actual performance and market conditions.

Stebbings and O’Driscoll close the show as Lemkin leaves for another engagement. The roundtable’s safety, legal and market discussions are the participants’ views at the time of recording, and the reported figures and future scenarios require independent assessment.