Open Source Catches Up, Regulation Can Wait: Sarah Guo and Elad Gil Answer Listener Questions

Guo says nothing in open source yet matches GPT-4, GPT-3.5 or Claude, and OpenAI remains out in front. But the landscape has shifted in recent months: Meta's Llama is quite good and many startups use it despite its licensing issues, Together has released a pre-training dataset, and Stability has released SDXL for images.

The larger dynamic, she argues, is that more teams now know how to train large models, and the cost of compute will keep falling. She cites heavy investment in distillation and claims from researchers that training the same size model a second time could be five times cheaper. Models can also annotate datasets and enable more advanced self-supervision.

On that basis Guo would bet on a GPT-3.5-level open-source model within a year, which she did not believe a few months earlier. Gil notes that this would still leave open source two to three years behind, and both expect a handful of closed leaders to stay one or two generations ahead. Gil says that with strong leadership, resources and talent, OpenAI is not something he would bet against.

Guo explains the recent buzz around autonomous agents: they orchestrate an LLM in an iterative loop toward a high-level goal, with planning, memory, prioritization and reflection. That is not a change to the model's architecture, but it allows new behavior. Her example is hackers trying to build agents that, given a goal like making money online, find demand and a supplier, set up a drop-ship store, generate ads and promote it.

Gil responds that the future is not evenly distributed: people in the AI community had described these approaches for a while, then one or two projects such as Auto-GPT implemented them and everyone took notice. Because an agent uses the context of its prior actions to update its next prompt, he says it was always going to be possible.

He is more interested in forms of memory. Each new ChatGPT session loses the context of earlier ones, so people are working on persistent context that remembers everything you have done with a system, or even what it has done with every other user. That would crowdsource an understanding of the world, which he jokingly calls a hive mind.

Gil asks why people want regulation at all. One reason is that incumbents benefit from lock-in, and regulators who block new efforts provide it. He points to the chart of price changes by industry: prices fell in lightly regulated areas like software and rose in education, healthcare and housing. A second reason is fear, but he notes that humans did plenty of harm in the 20th century without AI, and doomers have often been wrong, as with swine flu or 1970s predictions of famine.

Gil describes himself as a short-term optimist and long-term doomer who thinks AI may eventually be an existential threat but that the next 10 years will be fine. He would not regulate most things now. The exceptions he names are export controls on advanced chips, limits on AI driving offensive weapons such as drone swarms, perhaps global rules like those for chemical weapons, and eventually advanced robotics. He sees the 2024 election as a likely pivot point.

Guo agrees it is too early but lays out the two rational cases she has heard: very smart researchers worry about a hard takeoff in which models help train the next generation, and people are poor at reasoning about compounding. Others worry about a tactical reaction to a trigger like the election, either mass surveillance or lockdown, and would prefer a democratic process that gets ahead of it. She adds that alignment research is tied to capability research, so you cannot simply stop progress.

Gil separates technology risk from species risk. Technology misuse, such as shutting down a pipeline, is serious but recoverable, since you could turn off servers. For an AI to wipe out humanity it would need a physical form to replace the world that sustains it, which is why he would watch robotics. Guo agrees that embodiment is key and that we are far from it.

Asked whether AI investing hype is justified, Gil says he has lived through hype cycles in social, mobile, cloud and crypto, and that interesting things emerged from each. The difficulty is knowing which will work. During the mobile wave several photo apps went from zero to a million users in a week, but only Instagram stuck, partly because it combined filters with a Twitter-like follow network.

Gil quotes Peter Thiel's idea that you do not want to be first to market but last standing. Guo adds that a friend at OpenAI told her investors are wary at the application layer because it is hard to see what will survive, while they are enthusiastic about researcher-led foundation model companies. She notes that text-to-image was not an obvious killer use case a year or two earlier.

On opportunities, Guo names voice and dubbing for content providers and publishers, and compliance-style work such as turning a large bank's code into explanations for regulators. She also names audit, tax and accounting reconciliation, LLM-driven annotation, and the hard problem of personalizing LLMs with enterprise data, which involves data management, sync and access control. Gil agrees on voice synthesis and dubbing and says he is long on compliance, and he mentions Harvey in legal plus tooling such as LangChain, Chroma and LlamaIndex.

Gil thinks it may take about five years for the interesting application companies to emerge, a couple per year. The two also discuss Elon Musk's AI startup, with Gil noting that his call for a six-month pause looks self-serving given that he was starting an LLM company, though he says he would act the same way. They see room for vertical models, for example in healthcare where multimodal data may call for different training.

Gil says each technology wave splits value differently: roughly 80% startups for the internet, about 80% incumbents for mobile, and nearly all startups for crypto. He expects this wave to be about 80/20 in favor of incumbents, with Google a player, OpenAI aligned with Microsoft, and Salesforce likely still Salesforce.

Still, some companies could be vulnerable for the first time. ERP providers whose moat is integrations could face rivals who connect data to multiple systems in a day or two instead of six months of consulting. Gil also suggests private equity firms could use LLMs to cut costs in people-heavy businesses and bid more aggressively, though he doubts many are technology savvy enough.

Gil argues that people who do not know healthcare focus on drug development, but most early-stage spending is in the tens of millions out of a billion-dollar-plus total. He thinks AI will be applied faster to operations and services: delivery, cheaper doctor visits and telemedicine, and easing payments and insurance reimbursement. He cites Color, which he co-founded, and notes that market access is difficult.

Guo agrees on removing operational friction, such as prior authorization, where models can read data and fill out forms. She pushes back slightly: AI could change the drug cost curve not by reducing early research spending but by increasing the probability of being right, making expensive trials more efficient. Gil replies that animal-to-human translation and biological differences remain, and adds that regulatory capture lets incumbents drive up development costs.

Gil says no new biotech company outside Moderna has reached a very large market cap, around $30 to $50 billion, from a company founded after the late 1980s, whereas tech created dozens. He adds that the top four or five tech companies' market cap equals the entire biopharma industry. He notes that COVID vaccines and a nine-month penicillin effort during World War II moved fast when constraints were lifted, and concludes that most AI regulation now would be a bad idea.

Guo also stresses that governments can mobilize quickly, citing World War II aircraft production, and says winning in AI requires making compute a national priority. She recalls Gil's consistent advice to stick to tech companies over consumer-driven pharma. Gil says drug development could be hugely beneficial, but the open question is why no generational company has been built in so long, and how to remove obstacles while still protecting the public.