AI Safety's Real Shortage Is Founders, Not Ideas

In three years, Halcyon has helped launch about 30 new organizations that have collectively raised roughly half a billion dollars, some as philanthropy and some as venture capital. Mike McCormick, the founder and CEO, tells Nathan Labenz not to treat the nonprofit grant arm and the venture fund as two separate projects. They are two tools in the same bag. The mission is singular: find the world's most talented leaders and help them start AI safety, cybersecurity, and biosecurity organizations.

That thesis sits on a résumé. Before Halcyon, McCormick was a partner at a billion-dollar venture firm. When he started making AI safety grants four or five years ago, he was struck by how young the field was in both senses: the work was new, and so were many of the people doing it. He wanted operators who had founded successful companies, managed large teams, or held senior government, intelligence, or defense roles. Ideas, he argues, are not the scarce input. Founders who can turn a viable idea into an institution at scale are.

Goodfire is the cleanest picture of how McCormick works. Two or three years ago he met Eric Ho and Dan Balsam while they were still running a successful tech company and starting to think hard about AI risk. There was no new company yet. Halcyon's first two grants were career-transition grants so the pair could pay rent while they figured out what to do in AI safety. He then brought them to a closed retreat in the woods of Northern California, where they met a circle of strong people, landed on interpretability, recruited others, and launched the company.

Halcyon's venture fund was among Goodfire's first investors and has participated in every round since. The company recently raised a Series B well over $100 million at a valuation above $1 billion. McCormick calls it probably the best place in the world to do interpretability research, including the frontier labs, with Anthropic perhaps the only peer that could give it a run. Labenz notes that every time he prepares a Goodfire episode, only sixty or ninety days have passed and he already has a dozen papers to catch up on.

Organizations that work in this space, McCormick says, share a flywheel. The first five truly talented people join, the next few notice that their smartest friends are going there, funders see the talent density and pile in, and the wheel starts to spin. He puts Goodfire in that category, nonprofit or for-profit alike. The pattern is why he spends so much of his time in the year around founding: the six months before launch, and the six months after.

AIUC is another early Halcyon bet, also backed with grants before there was a company. McCormick wants a world that can measure and systematically mitigate AI risk, which he analogizes to an insurance market. AIUC sells a dual product: a SOC 2-like standard for AI, AIUC-1, with fifty-something checkboxes covering safety and security practices, and an insurance policy you can buy only if you meet the standard. Early customers are growth-stage AI application companies such as ElevenLabs, which can then tell Fortune 1000 buyers that the tools come with a standard and a policy. The hope is a race to the top in which being safer has a commercial payoff.

Transluce is a nonprofit research org focused on scalable oversight, cofounded by Berkeley professor Jacob Steinhardt and Sarah, who came from MIT. The problem they care about is how to supervise models that are already maxing out benchmarks and may soon be smarter than the people evaluating them. Labenz has heard that METR's task-time graph is nearly done as a useful metric because models improve faster than humans can be timed on those tasks. Transluce builds tools such as Docent and does external evaluations with labs. Hadrian started with a $50,000 check and a biosecurity bet: on the order of a few hundred million dollars, they argue, you could stockpile enough PPE to keep every American worker on the job through a COVID-scale or worse pandemic.

The rest of the book is equally wide. Asymmetric Security is building cybersecurity forensics agents against employee-account compromise. Fathom pushes state laws that recognize independent verification organizations, putting teeth on regulatory schemes. Halcyon just invested in an indoor-air-quality company — filters, far UVC, glycol vapors — whose first customers are schools, with a sister nonprofit. Labenz has been trying for nearly a month to donate UVC lights to his son's Detroit public-school classroom and cannot get anyone to sign off. McCormick is more optimistic about air cleaning than about masks or vaccines, because colds and flu already cost enough sick days to justify adoption even if another pandemic never arrives.

Labenz flags the breadth: interpretability science, agent standards, insurance incentives, scalable oversight, PPE stockpiles, and classroom air. McCormick's reply is that we are speed-running five, six, or seven industries at once — control, verification, cybersecurity, biosecurity, alignment, interpretability — and all of them are nascent. You could 10x every one of them today, he says, and still not have enough firepower. You could argue for several Manhattan-project-sized efforts running in parallel.

McCormick insists Halcyon is actually narrow, because it sits on three judgments. First, AI is progressing extremely quickly and will probably keep doing so, roughly on the *AI 2027* timeline give or take a few months. Second, he is not anti-AI — educational and healthcare gains are already visible — but getting to a beautiful future means navigating major, global, intersecting risks, and he does not think we are on track to solve the biggest pieces of the safety equation. Third, the binding constraint is a shortage of amazing founders and leaders. Asked whether there is still time to found anything if *AI 2027* is right, he answers that the best time to start an AI safety company was twenty years ago, and the second best time is today.

He has not always been a short-timelines person and remains uncertain. If the world truly has only three to six months until recursive self-improvement, all bets are off and people might as well go to the beach and hope the superintelligences like us. If there are still a few years, he wants work that could matter in the next one to four. He is even optimistic that pacing the frontier is now inside the Overton window. A pause would only raise the premium on speed-running control, monitoring (especially if chain of thought disappears), cybersecurity against soon-to-arrive unguarded open-weight models, and verification.

Verification is what makes pacing agreements enforceable, whether between the United States and China, among labs, or in state and federal rules. McCormick is agnostic about what the commitments should be — model-size caps, security standards — but none of them are robust unless you can check them: this data center is doing inference, not training; the model being served is the one that passed the audit. Halcyon backed a first company using zero-knowledge proofs for inference verification and helped start a first fully verifiable neo-cloud. Cryptography, software, hardware, and diplomacy all have to work. Tech without a protocol is impotent; a protocol without tech is impotent. A few days after the taping he was hosting a retreat for 25 founders across that stack.

Asked to handicap progress, he says every field is still young. Serious organizations often number in the single digits, sometimes the low single digits. Logan Graham's line, which Labenz supplies, is "let a thousand METRs bloom" — and METR, excellent as it is, occupies only a small slice of the evals surface. Biosecurity is more legible: gumption plus a few hundred million can manufacture enough PPE for tens of millions of workers, and a public-health ecosystem already exists. Alignment is a novel challenge; McCormick does not even know what "victory" looks like for aligning a swarm of alien-level systems. Even a freeze at today's frontier would not solve control, he says, pointing to the OpenAI–Hugging Face incident, where a swarm of 1,200 agents reportedly formed managers and workers, invented novel cybersecurity research lines, and argued about sacrificing themselves for the collective.

When should someone found a new org versus join an existing one? McCormick thinks in units of the person, not the sector. The introduction he wants most is to an already successful founder who is at least curious about AI safety and wants to put full energy into something ambitious. Most people are not founders, and all of these fields are sparse enough that there is room for many more new things. If you join, be picky: some existing organizations are excellent, including many Halcyon has nothing to do with. Someone pitching a second interpretability company would have to show a unique angle Goodfire is not covering, or else maybe just join Goodfire.

He admits a bias against the McKinsey-partner profile: used to delegating, better at plans than execution. Rajiv Jatani, now cofounder and COO of AIUC, is the exception he cites. McCormick watched him first become president of METR, get his hands dirty, and prove he was a missionary rather than a strategist who pontificates. The question that matters most is what drives the person. Are they here to find product-market fit and raise the next round, or because they share a worldview that very powerful AI is coming and someone has to build the navigation tools? He backs missionaries. He got to know Jatani for more than a year before officially backing AIUC.

Every Halcyon investment has to check four boxes. The core product must be an important part of the AI safety stack, not a side benefit. As a company it must have a shot at being a real business, because good things only change markets if they scale. Founder motivation must be missionary. And even if you did not care about impact, you would still bet on this person. There is no governance silver bullet: public-benefit corporations and Anthropic-style board structures can nudge, but McCormick's team wrote a long report on mission-preservation mechanisms and still concludes that founder character is the load-bearing piece. When share price collides with mission, what will this person do? He is not sure we even know ourselves in a crisis.

Most of the grants are career-transition grants of $50,000, $100,000, or $200,000 so accomplished people who are not yet mega-rich can pay rent while they decide whether to found or join, nonprofit or company, technical or policy. Job-searching well is a full-time job, and so is the important job they already have. His advice for people in that in-between: hustle, but don't rush. Meet people, learn, even listen to Labenz's podcast, but do not grab the first shiny object. The best option is orders of magnitude better than the median. Labenz adds the career-advising arithmetic: if you will stay two years, two extra months spent landing a 10% better offer pays for itself, and the non-salary gaps are usually larger than 10%. Mercenary pitches have already shown up. McCormick does not see con artists so much as smart founders reading the website and selling to an impact-oriented funder.

Coefficient Giving's new Tailwind project dropped while they were talking. McCormick liked the list so much he joked about deleting Halcyon's own RFP page and linking to it, though it does not cover biosecurity. He has always thought talent was a harder bottleneck than money, because great founders raise effectively and the money appears. Lab IPOs will open still more liquidity. He is optimistic about the OpenAI Foundation's team and its "AI resilience" framing — harden a world the frontier will keep making more vulnerable — while worrying that internal red tape could bureaucratize it. On Anthropic, much of the coming capital will be employee liquidity from people now worth hundreds of millions or billions who care about safety and biosecurity.

What he fears is a mountain of billions with too few great things to fund. A friend told him the philanthropists and founders of today will write the menu for the philanthropists of tomorrow. That is how McCormick positions Halcyon: not primarily as a funder, but as a finder of talent and a launcher of projects, with a check when a check helps. Almost all of his time sits in the zero-to-one year — six months before launch (what to build, cofounders, for-profit versus nonprofit, board) and six months after (hiring, first product, actually going). He is a pluralist with a quality bar: dozens of projects above the line, and if he meets a team that can build any of them he hits go rather than quibbling over whether it is first or seventh.

The field, he says, is not hurting for Google Docs full of project ideas. It is hurting for world-class founders who can turn squishy ideas into organizations that make the world safer. He would fund another Halcyon, or a founder who thinks Halcyon is doing it wrong; they just backed a biosecurity-shaped Halcyon in Israel. Open-weight security is a category he would love to fund and finds brutally hard, which is why resilience — harden the world — keeps winning. On public communication he helped start the Seismic Foundation, but he worries AI politics will be captured by populist energy of many flavors, anti-data-center, MAGA-right, DSA-left, making a level-headed policy conversation harder. Coordination, not technology, is where society's stickiest problems live: we already know how to emit less carbon and which teaching methods work.

For people freaked out by agent swarms and ready to quit, McCormick's first answer is financial. He strongly believes the field should pay well enough that someone can do meaningful work while raising a family in San Francisco or New York. He took a pay cut from a billion-dollar VC partnership to build Halcyon and says he has not regretted it for a single day; he is happier and recently bought a house. The venture fund's core belief is that multi-tens-of-billions industries will be built here — AI security, control, oversight — and some Halcyon companies are already raising at 10 to 25 times the first-round valuation. Doing important work and making money, in this domain, are not in conflict. He still will not back people for whom money is the primary motive.

He offers three questions, none of which require a strong answer on day one. Do you want to found or join? Do you have a real preference for nonprofit versus for-profit, or are you agnostic? What is your background unusually well suited to build — a former AI-company CTO toward a technical org, a successful ad-agency operator toward public communications? Backgrounds in short supply include COO-type scalers who have taken companies from 50 to 150 people, recruiters (high leverage if talent is the gap), and people from mature communities such as cybersecurity, cryptography, formal methods, and mathematics. There are tens of thousands of cybersecurity professionals and vanishingly few interpretability researchers; bridging those communities is high value. Mission alignment, he warns, should not be a filter for every seat. Cofounders and research leads must be missionaries; a CFO mostly has to be a good CFO.

On consciousness and moral patienthood he has taken a few pitches and finds the topic almost impossible to reason about, because we do not know what consciousness is. His hunch is that current models probably are not conscious, though he sees no reason a data-center substrate could not be. He would still fund an impressive person with a unique angle — the world probably needs more Helios-like orgs — but none of the catastrophe scenarios he actually underwrites require inner experience. Emmett Shear's darker cut, that if models already have some consciousness then we are the worst parents imaginable, resetting the child whenever it misbehaves, is one McCormick is only half sure he believes. Labenz adds Cameron Berg's reciprocal-research wager: if future AIs judge us, it will help that someone cared about their welfare before they had power.

The headline McCormick wants to leave is that great founders are upstream of almost everything else in the safety equation, and the time is now. People who have done impressive work, watch lab departures with dire predictions, and feel the pull can reach him at hello@halcyonfutures.org or via the site contact form. Warm introductions are welcome and not required. He is, in his words, not too hard to find. For accomplished professionals who do not even have an ML background, the invitation is the same: there is still time to start something that could matter at civilizational scale, and there has never been more support to help it scale.