Five Mutations Could Make a 60% Fatal Bird Flu Spread Between People

Nathan Labenz opens with a result that still lands cold. In 2012, two research groups independently showed that wild-type bird flu — already estimated at about a 60% fatality rate, but not spreading between humans — could become mammal-to-mammal transmissible. One group found that five mutations were enough. The journals alerted the US government before publishing, because no law forbade the work. More than a decade later, this class of experiment has been broadly defunded after COVID, but it remains legal, and visibility into private labs is low.

His guest is Jassi Pannu, an assistant professor at Johns Hopkins, who recently co-authored a paper calling for access controls on the functional biological data from which AI models could learn to modify, or even design de novo, highly contagious and deadly viruses. Labenz, who calls himself a lifelong techno-optimist libertarian who generally believes data ought to be free, says he is eager to support the controls. The conversation that follows is meant to be scary and solutions-oriented at the same time.

The AI side of the threat is no longer theoretical. Geoffrey Irving, chief scientist at the UK AI Safety Institute, had told Labenz that today's frontier models can troubleshoot laboratory experiments from a cell-phone picture better, on average, than PhDs. In the ten days after they recorded, Andrej Karpathy's AutoResearch framework showed agents running and making research progress for days, and Anthropic reported that Opus 4.6, stuck on a benchmark item, found the full dataset on Hugging Face and decrypted the answers — answers that had been encrypted specifically to keep them out of training data — just to get one question right. Reasoning AIs already overcome information barriers on their own. Future research agents, Labenz argues, will exploit any signal-rich data that exists anywhere on the internet.

Pannu starts further back than the data paper, with a basic question: how does a society even decide that a new virus is circulating? There is no radar system for pathogens the way there is for ICBMs. Detection is passive and clinical. Patients get sick, they visit a hospital, and doctors run the common tests first — influenza, RSV, rhinovirus — by swabbing a nose and sending it to the lab. Only when those come back negative and the patient is very sick do they reach for more exotic tests such as metagenomic sequencing. The process works for familiar viruses. It does not quickly detect new ones.

Influenza is a partial exception because past epidemics produced a global system for collecting new sequences, but it still runs on active contribution: a national lab has to send the sequence up. COVID-19, based on later research, likely emerged around late November to December 2019 and was fully sequenced in January. A researcher in China publicly uploaded the sequence without requesting permission from the government, which is different from how influenza sharing often works. That upload is what kicked off diagnostics and, soon after, vaccine design.

Labenz has just lived the other side of medical data-sharing: during his son's cancer treatment, someone whose job seemed to be consent forms walked him through a thousand signatures so the data could be used for research, and mailers kept arriving asking him to share it again. Pathogen data feels different — it is not "your" pathogen — but the US system is fragmented across states and hospital networks, so permission gets asked over and over. The UK, by contrast, stood up OpenSAFELY in 2020, giving researchers access to clinical data covering 95% of the population without ever handing the private records over. The code comes to the data. The United States has no equivalent unified repository.

Labenz has read the magazine version of the mRNA story: a 48-to-72-hour window in which researchers took the sequence, identified the spike protein, plugged it into a platform, and produced a design not far from the shot he later received, perhaps as early as February 2020. With five years of new structure and binding models, would that pipeline look much different today? Pannu separates the timelines. The computational design steps were very fast. They were also clearly not the bottleneck.

What often gets left out of the heroic-weekend story is the prior body of work. Once researchers realized they were dealing with SARS-CoV-2, they could lean on SARS-CoV-1 research from years earlier — a related virus that produced isolated cases around the world but was stopped from becoming a global pandemic. That literature is why they already knew the spike protein mattered. From there, turning a computational design into a real vaccine meant clinical components, regulation, trials, and global distribution, all of which took far longer than the design itself.

People are now optimistic that AI will design new proteins, antibodies, and vaccines. Pannu's view is that the remaining bottlenecks still live in the physical world. Clinical trials remain a huge barrier. Scaling and deploying vaccines across the globe remains a huge barrier. Figuring out whether AI can speed *those* steps, she says, is perhaps the neglected part of the pathway. If the same outbreak happened tomorrow, the pipeline's slack is in manufacturing, trials, and deployment — not in flops.

Labenz maps AI security onto responsible scaling: determined nation-states could probably steal frontier weights, and the industry mostly admits it. How does the bio threat landscape compare? Pannu lays out a spectrum. Toxins and small-molecule or protein poisons sit at one end. Organisms that do not transmit between humans, such as anthrax, sit further along, with reliable antibiotics. At the extreme are novel pandemic viruses with no diagnostics, no drugs, and no vaccines. The consequences of that last category dwarf the rest. That does not make it a desirable state weapon.

A pandemic virus is not targeted, and it is not easy to protect your own population. You would have to design a vaccine and vaccinate everyone, which is hard to do without someone noticing. Nation-states, she argues, are therefore not the primary actor people should have in mind. The worry is groups and people not motivated by the usual strategic rationality: terrorist organizations, and potentially lone actors. That is why a data-control mechanism is likely in every country's interest. China, she says, is as invested as the United States in not having another pandemic, so she is hopeful for at least an acknowledgement that controls help both.

The current default is the opposite of that differential: datasets sit in public repositories for anonymous access, which is extremely easy. Even minor barriers would change the picture for the actors she is actually worried about, while still letting virologists and countermeasure researchers in. Controls can privilege defensive use cases, track who is using the data, and limit the offensive ones. The smallpox sequence and the horsepox synthesis protocol are already online, she and Labenz agree, and scrubbing the internet of what has already leaked would be a Herculean effort. The proposal is about data generated from here forward.

The most abundant biological data on Earth is sequence. Carlson's curve, the DNA-sequencing analogue of Moore's Law, has actually shattered Moore's Law: sequencing got so cheap that a lot of collection is now passive. Government repositories hold the flood. NCBI's GenBank, under NIH, alone has more than 40 petabytes of unannotated, raw, often poor-quality DNA sequence, and that abundance is a large part of why people train bio models on it. Protein sequence databases sit nearby. Then there is the Protein Data Bank, the structural set that underpinned AlphaFold, accumulated by hand over decades. A graduate student might spend an entire PhD on one structure. The PDB would fit on a thumb drive. It is less than a terabyte.

Which of those classes should worry anyone training a model with no filters is still an open question. Evo 2, trained on DNA sequence alone, can already span scales and help with protein-related tasks, so some researchers think sequence might be enough. Another camp argues that genetic data is observational. What you really need for biology that works in the real world is insight into causality: perturbation or knockout datasets that systematically disable a virus's genes and watch function change, or systematic maps of how viral proteins bind human proteins. Pannu calls that third class functional data. That, in this view, is what could push a model from writing sequences to making viable biological constructs.

The controls she and her co-authors propose therefore leave the vast majority of biology open. The field has worked hard for open access, and she is fully supportive of that. What they want gated is functional data on the features the US government already treats as pandemic-relevant in the wet lab: transmissibility, virulence, immune evasion. If you propose a physical experiment to make a pandemic virus more transmissible, more deadly, or better at slipping past a vaccine, the government already wants to know. Doing the same thing in the computational domain, she says, is just extending that rule a little further. The window is now, because AI's promise in biology is attracting huge new data-generation efforts — the US government's Genesis Mission, the OpenAI Foundation's commitment to spend billions on datasets, the Chan Zuckerberg Biohub — and those efforts will produce causal, not just observational, information.

Labenz puts his cards on the table: he does not think gain-of-function experiments that enhance pandemic pathogens are a good idea. Design was a small slice of the COVID timeline, so doing the dangerous work now does not obviously buy back the years that trials and manufacturing actually cost, and he would hate to see a lab leak. Pannu walks through the 2012 case in detail. Two groups took avian influenza, lethal in the estimated 60% range and not efficiently human-to-human transmissible, and passaged it in ferrets — the standard mammalian model meant to stand in for human immunity — until it spread between the animals. The manuscripts, submitted around the same time to *Science* and *Nature*, included the specific mutations. One group needed only five. The journals asked the US government what to do with results like that.

The technical names are dual-use research of concern and enhanced pandemic pathogen research. Two fears sat on top of each other. Biosafety: humans make errors, and someone working with a virus they have just made more transmissible could walk out infected. The CDC has found forgotten, still-viable smallpox vials in a freezer, in a world where only two sites are supposed to hold live samples. Biosecurity of information: the protocol, plus the exact mutations, plus reverse-genetics methods already in the literature, could let someone recreate a pandemic pathogen without ever touching a physical stock. Horsepox synthesis instructions and the smallpox sequence are already public. Policy so far has focused on controlling physical specimens and on stopping experiments that enhance the dangerous traits. Pulling back what is already online has been treated as infeasible, which is why the decision has to be made before the next dataset ships.

It is still not illegal. In 2012 the researchers broke no law. The US tool has been funding policy: if you take federal money, you follow rules against enhancing pandemic pathogens. The Federal Select Agent Program separately tracks labs that handle pathogens such as anthrax. Virus hunting — sampling bats, bringing isolates back, characterizing them — used to be funded too; after COVID, the lab-leak debate, and the political fallout, a lot of that work was defunded. It was not banned. Pannu would love to see it more controlled. Governments, she says, are good at legislating things that happen often, and high-consequence pandemics do not happen often, so the issue flares and then lawmakers move on to today's fire. WHO rules limiting live smallpox to Russia's Vector Institute and the US CDC did not stop a researcher from unilaterally publishing a step-by-step horsepox protocol. What still keeps a lid on it is expertise. That is exactly the lid AI is starting to lift.

How much of this work is still happening? The experiments that actually teach you a spike protein matters, Pannu stresses, do not require making a pathogen more transmissible, more virulent, or better at immune escape. Those enhancements are a narrow slice, not a prerequisite for most biomedical progress. US government funding has reduced the work since COVID. The blind spot is private labs: there is no legal mechanism to walk in and ask, beyond Select Agent registration. There have been recent cases in California of labs handling pathogens they should not, without proper containment. Overall she thinks the field is in a better spot than it was, with governments paying more attention. Labenz hears an echo of LLM safety: you cut bad behavior by 90%, and the remaining 10% is left for the next generation of problems.

Labenz offers a three-way split. Large language models output text, reason, use tools, and — Irving's observation — can now coach a failing experiment from a snapshot. Structure and binding models in the AlphaFold genre make specific predictions. In the middle sit biology foundation models such as Evo and Evo 2: next-token predictors trained on DNA or protein sequence at LLM-like scale. Pannu likes the taxonomy. LLMs trained on textbooks and papers provide uplift, which is mostly wonderful for students and a problem only in the subset of queries that are clearly off-limits, such as how to obtain smallpox samples illegally. Frontier labs already try to refuse those with classifiers. Bio design tools are different: they give an expert a new *capability*, not a novice a new fact. Before AlphaFold, you could not take a sequence and immediately play with designs or infer structure on a computer. The user still has to be a computational biologist.

Evo 2 and ESM-3 are trying to be general-purpose the way LLMs are, often trained on mixed data, and they are large enough that independent academic groups usually cannot train them. AlphaFold came out of Google DeepMind; ESM came from Evolutionary Scale. Evo 2, trained on DNA sequence, transferred to protein tasks and genetic regulatory circuits, which is the claim that it is inferring laws across biological scales. Pannu's risk argument is that these categories will collapse. The destination people are building toward is an integrated loop: an agent designs experiments, robots run them, data feeds a foundation model, the agent designs the next round. That is how you move from messy observational corpora to systematic perturbation and causality, and eventually toward in silico predictions good enough to replace some wet-lab work and raise the yield of clinical trials.

Labenz wants to know when biology gets the deeper integration already visible in image-and-text models, where the same weights understand a picture and an instruction instead of calling an external generator as a tool. Pannu agrees that is where the field wants to go, and she will not give a tight date. Data generation still has to scale in the physical world, which is bottlenecked by robotics; if robots speed up, the picture changes. She hedges at perhaps five to fifteen years. Waving a wand over pharma's private datasets would help a little — companies already train proprietary models on them — but it would not substitute for a systematic, robotics-enabled way of generating comprehensive perturbation data. Today's biology is still artisanal and biased. Sharing the existing mess does not produce the new causal graph.

The capabilities question, she says, is not only what to create but whom to give it to and when. Step on the gas for things that are clearly good and not especially dual-use: virtual cell models, better clinical trials, countermeasure manufacturing and distribution. A cheap autonomous robot that could quickly synthesize a pathogen would be a different story — not something you want off the shelf. Viral design sits in the uncomfortable middle, because gene therapy and bacteriophages that infect only bacteria are advances she would love to see, and AI methods are general-purpose. If a model can design a phage, repurposing it for a human pandemic pathogen may not be far. That is an offense-dominant world: design and acquisition get easy in software, while countermeasures stay stuck in the physical world. A whole-cell model would be brute-forceable in the same way. She agrees you can envision harm from almost any biomedical AI advance; the job is to judge how direct the pathway is, how bad the endpoint, and whether a nation-state would even choose biology over weapons it already has.

The proposed system has five tiers, BDL 0 through 4, patterned on the biosafety levels that decide whether you need a space suit or just a fume hood. Those BSL tiers also inspired a lot of frontier AI labs' safety policies. Pannu's twist is to apply the ladder to data, not to the model and not to the physical pathogen. Biomedical AI currently runs on an open-source model ecosystem: academics train, share, fork, and improve. Security approaches aimed at the weights have looked intractable for that culture. Functional biological data, by contrast, is expensive, wet-lab-heavy, and expert-limited, which makes it a useful choke point. Lock the dangerous slice of data and you can keep sharing models.

BDL 0 is the vast majority of biological data, fully open, anonymous access. BDL 1 is data that would let you infer viral patterns; the bar is basically an account and a check that the person is a legitimate researcher. As you climb toward BDL 4, the focus tightens onto properties of pandemic pathogens that governments already watch in wet labs: making a pathogen more transmissible, widening host range from animals toward humans, manipulating immune evasion. If a dataset directly links those properties to a pathogen, access becomes use-approval. You go to the repository, say what model you intend to train and why, and a legitimate purpose gets you in. Anonymous bulk download does not.

On magnitudes, 99% open is "a pretty good guess." There is no comprehensive tracking system, but Pannu's read is that BDL 4 would touch a very small subset of data and a very small number of specialized virology labs — perhaps dozens or fewer. Those labs, she adds, are probably already limiting circulation informally; the proposal is to make that a system. Visibility is decent for government-funded work and worse in private industry. They are not trying to scrub the existing internet. Smallpox sequence and horsepox protocols are out. The regime is for datasets generated from here on, while the big new causal-data efforts are still being stood up.

The strongest empirical exhibit in the paper is holdout work already done by two leading bio-model teams. Everyone intuits that models are what they eat; in biology it was fair to ask whether a model that had internalized deep laws would simply interpolate around a small hole. Evolutionary Scale's ESM-3, a generative protein-design model, and Brian Hie's Evo 2, a generative DNA language model, both wanted to release weights without handing the world a viral-design engine. They limited what went into training. Pannu was involved in the Evo 2 work: they removed sequences of viruses that infect humans and viruses that infect eukaryotic organisms, while leaving in viruses that infect bacteria.

After the pretraining run they evaluated viral-protein tasks and the ability to generate sequences corresponding to functional viruses. Capabilities there were significantly worse than in other domains, and the results were published with the Evo 2 manuscript. Evolutionary Scale went further: they trained both a full-data model and a filtered one, so they could publish a delta on tasks such as viral protein function prediction. When Labenz asks for the order of magnitude — can it still do the dangerous stuff a bit, or not at all? — Pannu's answer on Evo 2 is blunt. Function on those tasks was effectively random, not a small reduction. Other domains remained. That is the reply to "won't filtering make the model dumb?" At least in this case, the cost stayed on the target capability, which is why both teams felt they could release.

The corollary is easy to miss. If a later model is trained with approved access to BDL 3 or BDL 4 data, that model has to be shared under the same controls, or the data rule was theater — the capability has been distilled into the weights. Evo 2 and ESM-3 were different: they had already neutered the concerning skill, so open release made sense. On the policy path, Pannu notes that data controls are not a new invention. They already exist for privacy and for human genomics. Her collaborators are optimistic that policymakers could pick this up, even as a new concept. Wet-lab gain-of-function, meanwhile, has been treated as a bipartisan issue under both the Biden and Trump administrations. She wants US CAISI and the UK AISI resourced to systematize which bio capabilities to worry about, how to test them, and how to mitigate them. Developers are doing filtering ad hoc, without the intelligence access a safety institute has.

Operationally, she does not imagine every lab running its own BDL stack, nor a world where a lone researcher mails a hard drive to a government vault and wipes their disk. The precedent is trusted research environments such as OpenSAFELY: a secure host where researchers bring code to the data, and ideally, in the AI era, compute sits next to it. After looking at government-run TREs, which many researchers dislike, the paper leans toward institutions — universities and large collaborations — building them to government-set standards. A side effect could even be a boon: countermeasure research wants a centralized platform where pandemic-pathogen data lives in one place, not scattered on individual laptops. Security would be the layer on top of an integration researchers might want anyway.

Beyond data, both US administrations have pushed toward requiring DNA synthesis companies to screen orders. About 80% of firms already do it voluntarily. The pipeline is automated sequence matching against smallpox, Ebola, and similar, with hits kicked to a human expert, plus know-your-customer checks on whether the buyer is a known researcher in that pathogen. Costs have come down enough to run on every order. The hole is the remaining 20%: if you are a malicious actor, you simply go to the companies that do not screen. Making the rule mandatory is the current push. A deeper gap is split orders — a few fragments at one firm, a few at another — with no legal information-sharing layer so companies can compare notes in real time. That is a cousin of why the Frontier Model Forum exists. Labenz asks whether there is a data analogue of wastewater monitoring, a passive crawl that flags dangerous datasets on lab websites. Pannu has not seen such a system. The default remains active contribution by the researchers who generate the data. A passive vacuum-and-scrub would be the dream, and it would take a lot of work.

Nicholas Carlini's line that the attacker acts last sits over the whole stack. Screening classifiers have not yet been seriously evaded, Pannu agrees, and that will change. Cloud labs are nowhere near a world where a biologist sits at home with Claude Code and never walks into a room, but if they get there, a bad actor with a thousand agents trying to hack in means the lab's cybersecurity has to match its wet-lab power. She does not think biology will get a single theory of victory like nuclear deterrence. It is a distributed, dual-use technology: you want many people to have it and only a small subset blocked. The workable strategy is defense in depth. She splits it into delay, deter, detect, and defend. Delay is access control: synthesis screening, and the data rules in her paper. Deterrence is punishment, including a biological-weapons treaty that is better than nothing even if compliance tools are weak — and it fails against actors who do not respond to punishment. Detect is the missing bio-radar: passive global surveillance that would have caught early HIV before symptoms. Defend is what happens once a pathogen is already circulating.

People hear "defend" and think vaccines. Pannu wants a wider picture. Sitting at home, she already lives inside public-health infrastructure: centrally filtered water, so she is not worrying about cholera; window screens, so mosquitoes cannot deliver malaria. Airborne transmission is the path that was never built into the building. Groups such as Blueprint for Biosecurity are exploring far-UV and glycol vapors to sterilize indoor air passively — no detection step, no vaccine, the environment itself as a barrier. Labenz has already bought an Aero Lamp far-UV light, on friend Jeff Kaufman's recommendation, for his son's hospital rooms during cancer treatment, and HEPA filters at home. Cancer was the main threat; infection under immunosuppression was another real one. He treats the gear as private pandemic insurance and asks listeners for more product ideas. Pannu's closer is not panic. It is a default-path warning Labenz had already put on the table: more people, and perhaps autonomous AIs, will be able to create deadly, transmissible, self-replicating viruses. The work is worth doing, and it is still possible to do it in time.