SB 813 Would Hand AI Safety Checks to Licensed Private Firms — If Government Can Watch the Watchers

Nathan Labenz frames the episode as the start of a short series on creative AI governance. The problem, as he puts it, is brutal: capabilities keep marching, timelines are uncertain, adoption is feverish, and countries are competing, while even tech-savvy policymakers could easily get the details wrong. Highly prescriptive rules of the sort Europe is attempting with its AI Act do not look, to him, likely to meet the moment. A laissez-faire free-for-all does not either. His exhibit is xAI claiming frontier capabilities while Grok 4 still self-identifies as Hitler.

The guests are Dr. Gillian Hadfield, Bloomberg Distinguished Professor of AI Alignment and Governance at Johns Hopkins, and Andrew Freedman, co-founder and chief strategy officer at Fathom. Their proposal, introduced in California as SB 813, tries to split two jobs that legislatures usually mash together. A government body — perhaps the attorney general, perhaps a new AI safety board — would name outcomes: systems must not enable bioweapons, autonomous vehicles must be safer than human drivers. Competing private certifiers would write the technical standards, check companies, and report results. Firms that opt in would get some liability protection when things still go wrong. Labenz, a lifelong libertarian, likes the attempt to bring market dynamism to governance. He also plans to press them on a race to the bottom, on what people give up in court, and on who has to stay competent for the system to work.

Fathom is a little over a year old, a nonprofit that went out with polling and qualitative interviews across industry and civic life. Almost everyone they met thought AI needed governance. Almost everyone also thought heavy-handed top-down rules would fail, and that leaving it to the labs would not serve society either. That hunt for a third way led them to Hadfield, and to people like Dean Ball. Freedman stresses that tech policy has too often been left to tech. AI, in their view, is a broader social repair job: the technology will break governance, the economy, and ordinary interaction, and someone has to help put it back together.

Hadfield has been worrying about legal machinery for decades. She started in access to justice, then in the mid-2000s turned to whether nineteenth-century lawmaking could keep up with speed, complexity, and multi-jurisdictional life. AI, she says, ramps that mismatch several levels. Her 2017 book argued that regulation itself needed new tools; the AI-specific paper with Jack Clark came in 2019, when she was on contract and he was policy director at OpenAI. Labenz notes Clark later became an Anthropic co-founder and policy lead. The public is meeting “regulatory markets” now. She has been building the concept for a long time.

Labenz quotes a line he cannot source: markets are not free or unfree; they have rules, and some rules work better than others. Hadfield calls it a constant refrain. Her PhD is in economics, taken jointly with a law degree. After the Soviet collapse, she says, economists assumed that lifting state control would make markets flourish. They did not, because contract, property, intellectual property, and basic regulation were missing. Countries without rule of law do not attract investment. Regulation, in this telling, is infrastructure that lets people participate with confidence, not a drag you subtract from an otherwise free market.

Top-down detail, Hadfield argues, gets less tenable as technology gets faster, more complex, and more cross-border. Markets are information-processing engines at ground level; legislatures, courts, and agencies run on another clock, and the laws and opinions have only gotten longer. You still want democratic control over which risks are acceptable — autonomous vehicles, companion AI, algorithmic decisions — but you can separate those judgments from questions like what data to train on or which red-team tests to run. Freedman’s first emerging-regulation job was Colorado’s legal cannabis rollout, a far simpler policy that still touched every corner of state law. The system’s strength was iteration: they did not see edibles coming, then they changed the rules. Predicting AI and writing guardrails that still make sense six months later, he says, is simply wrong. Independent experts who keep deciding what “good” looks like, applied universally, become the brass ring companies reach for. A statutory checklist, by contrast, becomes a floor tossed to the compliance department so the business unit can keep moving.

Freedman is careful: SB 813 is the fullest instantiation so far, it will be debated many times, and in his view it “requires quite a bit of revision.” The mechanism is still the point. Licensed third-party auditors, verifiers, or certifiers would have to prove they know current best practice and can check that companies meet it. They would scope their expertise — chatbots, vehicles — and developers or deployers would hire them because they want evidence they are meeting a duty of care. If something later goes wrong, that certificate is not quite a “liability shield,” he says, but evidence in court. Certifiers would keep returning to California with proof: certified cars crash less, certified chatbots cause fewer teen harms, and they beat both the uncertified status quo and rival certifiers. That is the intended race to the top.

Hadfield zooms out to a cartoon of twentieth-century regulation. Command-and-control says: install these scrubbers in the smokestack. Performance-based regulation, a later design, says: here is the pollution level at the top of the stack; you pick the technology. Regulatory markets take one more step. The government still sets outcomes — an acceptable accident rate for robotaxis, bioweapons capacity kept below a threshold that can be numerical or qualitative — and then deliberately grows a sector of independent firms whose job is to figure out the best way to hit those outcomes. She calls them regulatory services providers. They can be for-profit or nonprofit. They get licensed by demonstrating that their methods achieve the goal. Target companies pick from the approved list. In the original academic version, that purchase was mandatory, the way securities or auto-safety rules are mandatory.

The market idea, she insists, is to attract financial and human capital to a question nobody has answered: what actually works for AI — red-teaming, data review, embedded observers? Private actors already sit inside complex regimes as certifiers, auditors, lawyers, and accountants. The near-term problem is that the market barely exists, so you cannot turn around tomorrow and order every lab to buy an approved regulator. SB 813’s vision is to grow that market with a carrot rather than a mandate, the way a UL mark became valuable before it was written into code.

Asked for a living analogue, Freedman points to Underwriters Laboratories. Pick up almost any electrical product and you will find a UL mark. The firm began at a late-nineteenth-century world’s fair, when organizers brought in independent experts so the grounds would not burn down. The business spread; consumers did not have to trust every gadget if they could demand the stamp. About a hundred years later, UL started getting written straight into law. Freedman does not want to wait that long, and he does not want a purely private ecosystem whose true north is covering labs’ legal backs. Hadfield’s race-to-the-top design, with goals set in public legislation and real accountability, is what he thinks can supercharge a UL-like function, keep it from industry capture, and point it at the public good.

Hadfield agrees the full model does not exist yet; the proposal is legal technology as innovative as the underlying tech. Plenty of things come up to the doorstep. ISO and other nonprofits already sell technical standards that markets or governments then pick up. Closer still is medical-device quality: a consortium of about five countries, including the United States and Canada, lets each choose its own standard — FDA in the U.S., ISO in Canada — then publishes a shared list of authorized certifiers. A single audit by one approved certifier unlocks sale in every member country. Securities regulation began on private exchanges that demanded disclosure; government later absorbed the practice. Today FINRA is a private membership organization whose rules the SEC approves. Public-private overlap is already the real regulatory system, she says, not the command-and-control cartoon. Regulatory markets lean harder into government’s outcome role and into market activity around how to get there.

The original papers imagined a mandate. Because the certifier market is thin, SB 813 instead tries to create a UL-like incentive: you need not participate, but if you do, and the state has approved the overseer, you can argue in a tort case that you met your duty. It can even become a formal presumption of compliance, because the state would only approve bodies whose methods, if followed, satisfy that duty. Freedman likes tort as the backdrop for two reasons. It forces the market to admit where harm is actually expected — if frontier labs think they will never be sued over some risk, that slice of the market is worthless. And common-law duty of care, though it varies by state, is national enough that meeting it in California can travel. Combined with private certifiers, that could centralize practice across states, even internationally, instead of a compliance patchwork.

Labenz asks what the public gives up compared with cars or drugs. Freedman’s first answer is that nobody knows, including the two of them. Tech has been relatively shielded from tort; agentic systems that act in the world will not be. The nearest proof is the Character.AI case, which had just survived a motion to dismiss. He expects state judges around the country to find varied liability for developers, deployers, and applications — the full stack. As drafted, SB 813 offers a *rebuttable presumption*: certification counts as substantial proof you were not negligent, and other evidence can still knock it down. They are open to moving that needle. Tort so far is “potential energy,” not kinetic; he wants companies thinking about best practice now, not in ten or fifteen years.

Hadfield, in law-professor mode, tells students they can always sue. That is how common law evolves: case by case, after harm, through an expensive process that does not give everyone access. FDA approval does not bar a drug suit; auto regulation does not bar a suit against a manufacturer. Courts simply weigh compliance as evidence of reasonable care. True lawsuit bans exist — vaccine injury compensation, the September 11 victims’ fund that closed the door on airlines and the Port Authority — and this proposal is not that. The point is to pull the definition of “reasonable steps” forward, before release, with an independent approved overseer, instead of waiting for accidents and lawyers. Freedman’s true north is fewer people harmed, not more cases to try. Labenz notes Waymo and Swiss Re charts that already compare robotaxi injuries with human drivers. That subcategory looks clean. Exotic, poorly labeled “agents” do not.

Labenz spent time in financial services before the mortgage crisis and watched credit-rating agencies become captured and, in his view, worthless, just as exotic products exploded. AI, he says, is another explosion of exotic products. How do you get a durable race to the top instead of 2007? Hadfield and Clark discussed rating agencies in the 2019 paper because everyone reaches for the analogy. The important facts, she says, cut the other way. Government created demand by requiring ratings to issue bonds, then immunized the agencies from liability for those ratings. Oversight of how well they did the job was effectively zero. Regulatory markets are supposed to do the opposite: move government from impossible lab-level detail to muscular oversight of the private regulators, with licenses yanked if they miss the standard.

Imagine a goal of state-of-the-art protection against giving bioweapons capacity to someone with high-school chemistry. Competing certifiers then have a market reason to tell the government, look what we figured out, and by the way our rivals are pulling the wool over your eyes. That can raise the standard and hunt for cost-effective methods. Current AI governance, she argues, has defaulted to untested process: logs, internal officials, box-checking nobody validated. A race to the top would make those processes falsifiable against an outcome. Freedman adds two hard constraints. Government must get into certifiers’ finances so labs are not seating the funders, and so a firm that would go under if it denied four of five labs can still say no. And government must get real about outcomes: some have human baselines, some are new harms without incident reporting, and some first incidents are so catastrophic that “we followed the rules on the way there” is no defense.

That last category is why they want SB 813 amended. Labenz raises bio-risk on the scale of a pandemic: you cannot sue the Wuhan Institute of Virology into making ten million dead whole. Freedman agrees some harms are too big to sit inside this mechanism and should be carved out. Hadfield is equally plain that the tool is one instrument in an already overlapping system of tort, standards, corporate incentives, press, and international regimes. AI is also not “another product” like a car or a drug. It is a general-purpose technology that will hit education, health, justice, logistics, city management, bioweapons, and companions, each with different goals and methods. You will not “regulate AI” once. What markets can do, she says, is sniff problems at ground level — an enterprise that will not plug a hallucinating chatbot into customer service until someone offers a credible assurance; parents already in tort over companion models and children. Sitting in a boardroom with a whiteboard and listing every AI risk is, in her word, a fallacy of lawmaking.

Labenz puts on a cynic’s hat. Frontier labs, relative to grim counterfactuals, are more responsible than they might have been; follow the incentives and they will still pick the most permissive, cheapest certifier on the menu. If money flows from labs to the “MROs” in SB 813 parlance, getting picked is how you get paid. In the current bill, the California attorney general approves the organizations and has to keep watching them. One AG to the next can mean a crack team or a shop open to lobbying. Is the public really putting that much trust in one office? Freedman says yes, and that it probably has to change. He wants a commission with expertise, ongoing tracking of outcomes, and a “fear of God” that a certifier’s license can disappear after a complaint if there is money to investigate. If several states or countries license the same firms, one withdrawal becomes a signal to the others. Numbers go public: a certifier whose “we made the world better” stats are laughable can be compared with one that is actually moving outcomes. If government is simply uninterested, every structure fails. He still does not know another design more set up to turn ceilings into a qualitative race to the top.

Hadfield tightens the point. The system is only as good as the state’s capacity to oversee private actors — the same way the IRS, FDA, or SEC is only as good as its rules and enforcement. The grand proposal is to shift government effort onto those private bodies. That is the backstop against “come on over, you will not have to do much.” Capture is a problem in every regime; the comparison is not to a fantasy of perfect agencies but to the status quo. Right now, she says, government has defaulted to corporate self-governance: labs tell us which red-team tests to run, visibility is limited, industry standard bodies are corporate-funded. It has defaulted because the technical problem is hard. The point of the model is to handle that difficulty without giving up a more muscular democratic role: society still names how much risk it will take. Labenz had said we are flying naked. She agrees.

Labenz quotes Dean Ball: republics require virtue. Someone in a key seat has to try to do a good job or any design goes to hell. Hadfield adds visibility. This may be the first time a general-purpose technology of this consequence has been built almost entirely inside private companies, wrapped in a legally created ring that keeps the inside in unless the firm talks or the government forces a look. Governments cannot regulate what they cannot see. Independent red-teamers and robustness shops are already appearing; she wants to lean into that sector, attract investment, and make it government’s partner. Dollars should move, including to nonprofits now under-resourced on lab contracts. Two private parties under contract, she thinks, will share finer-grained information than a lab will give Sacramento, because confidentiality and IP feel safer. The public regulator then sets whatever disclosure it needs from the certifier: show us what you learned, show us your method hits the target. Layered visibility, not a raid on every server. Virtue and visibility, together.

Labenz channels Matt Perault, a16z’s head of AI policy, who worries the rules become so onerous that only big tech can enter the beneficial regime, then startups cannot compete or raise capital. Freedman will not bite that bullet. He cannot think of a structure more able to scale different solutions for “little tech” than for frontier labs. Guiding language in SB 813 could create a specialty lane for smaller models, apps, or deployers with less immediate risk and a much lower burden than a one-size SOC 2. A ten-person team shipping a limited application is not a 10,000-vehicle fleet that must stop for a child in the street. Top-down FLOP thresholds, he says, will not make sense tomorrow; specialist certifiers can move. In a no-regulation world, startups also lose: only incumbents can prove safety to a bank. A tiny team that wants to remake banking has no way to be trusted. Hadfield goes further. She has been driven for decades by legal systems that became too expensive and slow, heavy on unvalidated process. GDPR’s logs and definitions are now under pressure in Europe as a drag on startups. The goal of this mechanism is cheaper, more effective regimes, and she says society needed the shift ten years ago. AI is accelerating while “we still have our shoelaces tied on the starting line.” Labenz likes the implicit re-evaluation — not quite a sunset clause, but a meta-structure with new entrants that might age better than most AI bills.

The second objection is from law professor Gabriel Weil, who Labenz will interview separately: expand liability to near misses so catastrophic harm can be deterred before it happens. Hadfield will not treat litigation as the first line for nuclear-scale, pharmaceutical-scale, or even financial-collapse risk. The FDA does not wait to see who dies; it demands safety and efficacy up front, with lawsuits as backup. Trading agents that could cause the equivalent of bank runs belong in the same bucket. Tweaks to tort may still be worth making. They are not a substitute for oversight. Freedman worries about the downside: near-miss liability teaches firms not to report, to silo knowledge so “we didn’t know,” and to skip red teams because the more you know the more you can be sued. He wants people rewarded for being proactive, not only sued after the world has burned. Labenz, listening, softens his own framing: expanding liability and offering compliance protection may not be as diametrically opposed as they first appeared.

Why not just require insurance and let the best risk-pricers in the economy do the work? Freedman is most annoyed, in a nerdy way, by waving a magic wand labeled insurance without asking how insurance actually works. To price and charge for risk you must know what the risks are and what mitigates them. That knowledge base does not exist for AI. Insurers are not better guessers than anyone else; a mandate could produce self-insurance at astronomical prices. Hadfield adds that auto, construction, and pharma insurance price against a thick backdrop of codes, cases, and agencies. We do not have that. Insurers will not become AI regulators, and it would be undemocratic to let them decide how much bio-risk society will accept. Her University of Toronto institute’s report treated insurance as a complement and another carrot: if you implement this method, you can get a policy. Armilla, a regulatory-technology startup she uses as a go-to example, partnered with Lloyd’s so that using Armilla’s approaches unlocks coverage. Technology first, then the policy. A bad event is not even a “risk” to the company unless courts or regulators attach liability. Without that, the market is not massive; it is undefined.

Fathom itself does not plan to become the private regulator. It is a philanthropically funded nonprofit with no equity endgame. Freedman wants proofs of concept, small grants to technical partners, and a picture of “what good looks like,” because nobody has time for a two-hour podcast. He also does not trust insurers’ governance on the worst tail: they might happily cover a catastrophe so large that, if it hit, there would be nobody left to sue and no society left to pay. Their focus will be the next tangible claim they might actually be on the hook for. The important question is how third-party certifiers are accountable to the public rather than to a perverse economic motive. Hadfield’s close is to stop waiting for a perfect structure. Society needs an MVP of new regulation, lots of people poking the model, and governments that set acceptable risk while overseen markets hunt the methods. SB 813 will keep evolving. The path has to start.