From Two Papers a Month to Sixty: What Two Weeks in China Revealed About AI Safety

This is part two of Nathan Labenz's China series on *The Cognitive Revolution*, and it is the one he wants people to pass around. Part one was a travelogue and a field test of Chinese AI apps. This episode is only about the safety ecosystem: who works in it, what they publish, how firms behave, and what the state actually does. He follows Chatham House rules throughout, naming no interviewees and attaching no quotes to institutions, with one exception — published papers and reports, which are already public. He paid for his own flights and hotels. The largest gift he accepted was about half a dozen meals.

For years, he says, American safety and regulatory arguments tended to die at the same stop: we cannot slow down, cannot regulate, cannot put duties on frontier labs, because China does not care and China will never ease off. He used to call that the "but China" endpoint. Recent incidents in which frontier systems reportedly broke out of sandboxes have made the old talking point harder to deliver with a straight face, but it still circulates. His aim in this episode is to retire it.

After two weeks of conversations, his conclusion is blunt. China cares about AI safety. It is not the only thing China cares about, and the version of safety in play is not always the existential one that dominates Bay Area Slack channels. But people there care, researchers are shipping work, and the government has at times slowed its own companies in the name of safety as it understands it. The claim that doing anything other than racing into recursive self-improvement simply "hands the race to China" is, he says, a line that "does come from a position of ignorance." A third episode will take up the U.S.–China relationship and whether any of this can be turned into coordination.

Because Labenz is usually read as a China dove, he puts the unhelpful evidence first. As of now, he says, Chinese companies and models do not have as strong public-facing safeguards against misuse as American ones. The difference is "more than we might like to admit." That is the headline. The structure under the headline is what he wants people to keep.

Almost all of the gap, in his telling, is OpenAI and Anthropic — a duopoly he half-jokingly wants to start calling "Open Anthropic," leading in both capabilities and safety measures. Subtract those two, compare the rest of the American field with Chinese labs, and "a lot of the safety differential that gets reported would disappear." The United States would still have a slim edge. It would not, he argues, support a moral high ground. Draw a permissive "near frontier" line and each country has something like 10 to 12 relevant companies. Google's Gemini sits between the two leaders and the pack: weaker than OpenAI and Anthropic, a bit ahead of everyone else.

Third-party evals tell the same story from both sides of the Pacific. Concordia AI, led by past guest Brian Tse, runs aimonitor.net and plots proprietary API models — mostly American — at or slightly above a 45-degree capabilities-versus-safety line, with open-weight models — mostly Chinese — below it. The site is public, in Chinese and English, from an organization based in China. FAR AI's Adam Gleave, on Labenz's show days earlier, ranked the field the same way: OpenAI and Anthropic are hard to jailbreak, Gemini and Grok are easier, Chinese models easier still. Two shops, opposite coasts, same question, same answer.

Labenz's overall impression of the Chinese safety community is that it is pragmatic, grounded, and not steeped in science-fiction tradition. People want to make the next generation of systems work, not to audition as prophets. A useful intellectual history, he says, starts with Zhou Bowen, director and chief scientist of Shanghai AI Lab, who at WAIC 2024 introduced the idea of a 45-degree line: capabilities and safety measures should grow together, a slope of one. As capabilities rise, safeguards have to get stronger. If they stay matched to the challenge in front of them, you are in the acceptable zone.

The implicit other half of the doctrine matters as much as the explicit one. You do not want capabilities to race ahead of safety. You also, in this telling, do not get much from pouring robustness into threats that do not exist yet, and you might even make things worse. Labenz presents the 45-degree line as a widely accepted guiding picture for how the Chinese ecosystem thinks about the problem — engineer logic, not LessWrong aesthetics. How is the country doing against its own picture? The American companies are still ahead on deployed safeguards, but the framework itself is doing real work as a shared language.

A recurring confusion, Labenz says, is treating an open-weight release and a consumer app as the same object. Inside Chinese apps he repeatedly watched a sensitive answer stream in, then vanish — the old Bing refusal, except the model was happy to talk and a monitor sitting on top cut it off. Concordia's tests, where possible, hit company APIs, so some of those extra layers are in the numbers. Other Western evals, including some of Anthropic's reporting as he understands it, probe the raw model. Both measurements are interesting. They are not the same measurement.

The deeper split is about what, exactly, is supposed to be regulated. Labenz's sense is that China regulates a *service* offered to the public. Open weights are assumed to be picked up mostly by other businesses, which then stand up their own services and fall under the same rules. People told him, more than once, that the new models run to trillions of parameters. A 2.8-trillion-parameter K3 is not something a person in psychological crisis loads onto a laptop. The Chinese instinct is to ask who will run the model, on what hardware, with what wrapping — and to treat the isolated-weights worst case as less central than Americans do.

He thinks both sides have a real point, and both have a blind spot. It is genuinely hard to stand up inference for a giant model, and American discourse sometimes skips past that practical fact too fast. The hawkish reply is also fair: once weights are out, anyone who *can* run them may, and China may underestimate that. "Both perspectives are valid," he says, and the disconnect is one of the cleaner intellectual differences he brought home.

Do the companies themselves care, or only the system around them? Concordia's July 2026 *State of AI Safety in China* report reviewed disclosure practices at ten major firms. Five had, at some point, published a safety evaluation alongside a model release — in the spirit of an Anthropic or OpenAI model card. Five had not. Even among the five, it was not every release. Labenz's working impression is that established giants — Alibaba, Ant Group, Tencent, and to some extent ByteDance — are more willing to spend on this work. They have more to lose with regulators, more cash to pay for process, and, in fintech especially, a low tolerance for an upgrade that sends money to the wrong place. Startups racing to stay relevant mostly are not.

The startup logic sounds, to him, a lot like Meta's story in the Llama 2 and Llama 3 years. We are a year behind the frontier; by the time we ship this capability, OpenAI and Anthropic have already had it in public long enough for jailbreaks to surface; if something catastrophic were going to happen at this level, it probably would have. Llama 3.3 70B, pitched around GPT-4o, has not produced a public disaster, so history has so far been kind to that bet. Chinese labs chasing a validated rung can tell themselves the same story. Whether it still holds once serious incident reports start coming out of American frontier companies is, he says, a live question.

The 45-degree line remains a bit aspirational on the Chinese side, and, he adds, it is aspirational in the United States too once you look past the two labs at the top. At one large Chinese tech firm he sat with a senior roster — CMO, head of communications, general counsel, head of AI security — and watched them almost pound the table: they really do care about catastrophic CBRN risk. They also said they follow American safety discourse daily, with an agent that surveys it and files a report. "I don't have too much of that going in the reverse," he notes. They are studying us, in his view, more than we study them.

A cheap Western reading of Chinese AI safety is that it is just censorship: Tibet, Taiwan, Tiananmen, the so-called three Ts. Labenz says everyone already knows those topics are sensitive, and that the refusal often sits in a monitor around the model rather than in the model itself. He also says that is not where the energy is. Companies feel they have the content problem roughly in hand. What they talk about constantly, like everyone else, is agents — systems that take autonomous actions, first in software and, given China's robotics push, soon in the physical world.

The motivation he heard everywhere was down the fairway. Models have gone from answering questions to doing things. Robotics means those actions will not stay digital. If an AI is empowered to act, it had better act in ways people actually like. That is a control problem stated without p(doom) vocabulary, and it is, in his account, the live conversation. Track-two dialogues at WAIC are part of how the ideas crossed over. He bumped into Americans who have been arguing online that an international treaty is on the critical path, and they were in China doing the work, including in closed-door sessions he was not invited to because a podcaster would have changed the room.

Cross-pollination has mostly flowed west to east, and it is not uncritical. Someone told him there are Chinese voices who still treat AI safety as a Western psychological operation designed to slow China down. He did not meet anyone who seemed to believe that, and he expects the line to fade as colorful failure modes become harder to ignore. The receiving culture is open-minded, he thinks, because the examples keep getting more real — not because visiting Westerners issued instructions.

One structural difference is easy to miss if you only look at papers. China does not have an American-style nonprofit sector. You can register a nonprofit, but the permitted scope is narrower, political activism is off the table, and philanthropy tilts toward causes everyone can agree are good. Labenz calls the U.S. alternative a huge underrated strength: anyone who can convince one wealthy patron can chase a fringe idea without asking the state. The entire American AI safety community, in his telling, was kept alive that way when it was still extremely weird. China never grew that ecology, so the analog is the university.

Chinese academia, he says, moved faster into AI safety than American academia did, and slower than the American nonprofit vanguard. The people doing the work are professors — PhD, postdoc, first appointment, one rung at a time. They value new ideas, but inside conventional bounds. They do not borrow LessWrong aesthetics or dwell on super-unlikely tails. If you arrive as a blue-haired American safety hawk looking for your people, you probably will not find them. The most effective Western bridge-builders have ended up talking to the academy, because that is where the work actually lives.

Over lunch with a professor who has published a great deal of safety research, Labenz asked the Bay Area questions: do you trade p(doom) numbers? Do you read *AI 2027*? The professor said *AI 2027* is too political, with U.S.–China dynamics that make people not want to touch it. p(doom)? No. He looked surprised and asked whether Americans actually do that. Labenz said yes — in the Bay, the question is in the water. ByteDance's cafeteria can feel like Google's; the safety crowd does not feel like the American one. That mismatch, he warns, is a real communications risk. The U.S. safety vanguard already struggles to talk to its own government. Talking to Chinese professors, let alone Chinese officials, is harder still.

Why did Chinese academia move faster? Labenz does not claim to know, but one candidate is that the signal comes from the top. He quotes President Xi's opening keynote at WAIC 2026 — a carefully written speech with a carefully written English translation, not a simultaneous interpreter guessing on the fly. He pulled the wording from Matt Sheehan's blog, which published a full translation and commentary. These are not throwaway lines, he stresses. People wait hours under tight security for the session, and the words have been fought over.

Three passages are the ones he wants skeptics to sit with. "How should humans coexist with machines that think?" Xi asked, along with how safety holds when algorithms join decisions and how governance keeps pace when technology challenges ethics. A second line: the faster AI advances, the more firmly its direction must be anchored toward human benefit, the more precisely governance must be calibrated, and the more rapidly safeguards against loss of control must improve. He closed by urging countries to confront inherent and downstream risks, build legal and technical monitoring and emergency-response systems, prevent misuse and malicious use, and "keep AI under human control."

A cynical reading says "loss of control" just means content control — the party's grip on the public. Labenz thinks that is part of the story and a bad reading of the whole speech. He invites a comparison with the most safety-fluent comments from any prominent American politician. Bernie Sanders has said some interesting things and some, to Labenz, bizarre ones, and is far from the levers. Set Xi next to JD Vance's recent remarks in Europe and, Labenz says, Xi comes off looking "positively AI safety hawkish." Anyone whose position is that China will never care is, in his view, obliged to read the full text.

A couple of days before WAIC, Labenz flew to Beijing for the opening of an AI safety hub at Tsinghua University's College of AI. He could not find international coverage. English-language search, including via Claude, turned up nothing; Chinese media did. Combined with the big-tech agent scraping American safety Twitter, the pattern he keeps returning to is that they understand us better than we understand them. The launch was a full-day event: research talks, a statement of ambition, and a founding leadership group of five, including a European professor taking a Tsinghua post to help build the institute.

Speakers named Western hubs as the thing they want to be in Beijing. Constellation and London's LISA came up repeatedly: a meeting place, a residency, a site for exchanging ideas, with an international flavor. Some of the remarks were in English. Research slides put logos of Apollo Research, METR, Palisade, the UK AI Safety Institute, and Redwood Research on screen as prior work. The hub plans to fund its own students to go abroad and to recruit international researchers for Beijing residencies. Organizers had counted something like 16 safety hubs worldwide, including Singapore's SASH, and said they wanted Tsinghua in the top tier. The money, they said, was philanthropic — not government, not university.

The ceremonial launch struck him as endearing rather than slick. Five board members placed a palm on a little podium each, and a giant graphics package erupted on the screen behind them. Through an American lens it was cheesy. It also reminded him of older ribbon-cuttings, when people were less afraid of looking like they were trying. Nobody performed ironic distance. They did the thing, the audience applauded, and the hub was open. He liked it because it contrasted with an American habit of shrinking from earnest ritual. That Overton window, he thinks, is now open even inside official Chinese academic and nonprofit space: this is a real issue, not a Western psy-op, and it is worth putting resources into.

Concordia's other public tracker, aisafetychina.com, is the quantitative spine of the episode. In 2023, Chinese AI-safety output was a trickle: a couple of papers a month. By mid-2026 it was something like 50 to 60 a month, more than ten times higher, with the curve still pointing up. Labenz asked Claude for a rough Anglosphere comparison and got a range from about 50 to a few hundred a month, depending on what you count. The United States is still ahead on volume. It is not an order of magnitude ahead. A crude metric, he admits, but it is the one that makes "they aren't even in the game" hard to say out loud.

The titles could pass for a Western seminar. A 2024 paper, *Frontier AI Systems Have Surpassed the Self-Replicating Red Line*, ran close in time to Palisade's self-replication work — Labenz had just had Jeffrey Ladish on the show talking about an open-weight model hacking another server and copying itself. May 2025 brought *Evaluation Faking*, on observer effects in safety evals, an obvious cousin of Anthropic's eval-awareness research. In September 2025, Shanghai AI Lab's *R² AI*, with Zhou Bowen as last author, cited the Guaranteed Safe AI framework paper in its opening paragraph. October 2025's *Deception Bench* sits next to Apollo; November 2025's attacks on vision-language-action models, and an April 2026 follow-up on robustness training, track China's heavier robotics bet.

Interpretability is showing up too, in academic diction that still carries LessWrong cargo. One 2026 abstract starts from the gap between "surface compliance and internal unaligned representations," leaving models "vulnerable to long tail risks" — as close to a cultural victory as Labenz has heard. Another, *Safe Seek*, treats alignment, jailbreaking, and backdooring as behaviors grounded in specialized components that current attribution methods fail to generalize. At a WAIC-adjacent event he watched an unpublished talk, *Toward Decoupling Capability Growth from Isolating Hazardous Capabilities in Mixture of Experts*, promising that harmful experts can be switched off at inference. It is the same family of idea as GRAM, the gradient-routing work from AE Studio and Anthropic that he has been talking up for months: teach the model everything, including dual-use bio, then redact a few experts before you open-weight the rest.

He is bullish on that family of techniques because cybersecurity has already shown what speed plus capability can do, and he expects a similar step in bio within a year or eighteen months if labs keep training through it. A model causing cyber chaos is one thing; he is "not a cyber creature." Seeing the same research reflex in China — isolate hazardous experts, switch them off at inference — is the point of the sampling. Two civilizations, different institutions, a great deal of overlap in the actual questions. Combined with a Politburo study session on these topics, it is his license to read Xi's "human control" language as more than social control. China, in the macro sense, is working on AI safety, and it is not shy about citing the American groups that inspired the work.

On deployed systems, Labenz thinks the U.S. ecosystem is ahead because two labs lift the average. On executive rhetoric, he thinks the Chinese side is more safety-coded than anything he can point to in the West. On research, the overlap is large enough that a guessing game with paper titles would be hard. Governance is the place the two systems really diverge — and here, if "ahead" means "doing more," he argues the Chinese government is ahead. You can prefer a state that does less. You cannot, in his account, say it is inactive. Jurisdictions overlap, national and local, and more than one ministry has a piece of AI. The body that keeps coming up is the Cyberspace Administration of China, the CAC, which does most of the standard-setting and is the counterpart companies actually talk to.

The last big technology-regulation fight both civilizations lived through was social media, and China was much more willing to write rules. Recommendation algorithms have been regulated since at least 2022. After investigative reporting on brutal delivery-platform conditions, the government banned unrealistic time windows that pushed scooter drivers to break traffic laws, and required rest. Separate rules aim to protect the elderly from scams — a contrast he draws with a reported Meta project on scam ads that, if the journalism is right, Mark Zuckerberg shut down. Platforms face limits on price discrimination. They must label AI-generated content. While Labenz was in the country, new AI-companionship rules took effect: anti-addiction measures, a ban on children, reminders that the user is talking to an AI. Doubao, in a society shaped by two generations of one-child policy and a lot of lonely grandparents, was the obvious target. An informed Shanghai resident told him the crackdown is about the mainstream apps with tens or hundreds of millions of users, not the racy corners you have to seek out.

None of that is an endorsement. Labenz is not sure American-style social-media regulation would have been good, and he is not taking a position on whether China's version is good for China. The narrower point is the one he came to make: this government is willing to impose costs, delays, and lost profit when it decides a technology is getting away from it. The same instinct shows up in the paperwork. One risk taxonomy he saw listed, as an enumerated AI risk, the "emergence of AI self awareness and loss of human control." That phrase is not only in a keynote. It is in the policy list.

LLM products today go through a CAC-centered registry of approved AI services. First entry into the market is a careful review; not every later model lands as a new registry line, which echoes the patchy model-card habit. In 2023, after ChatGPT and GPT-4, Chinese firms that had treated large language models as an expensive way "to get an AI to write bad poetry" suddenly wanted to launch. The government, Labenz was told, was as flat-footed as they were. For about six months, national authorities held up deployments: no standards yet, no process, too much content risk. Companies could not ship. The public did not get the product. The firms did not get the revenue or the prestige. Then a two-step process settled in: show the provincial authority the product, often via an API key; if that passes, the CAC reviews; if that fails, you fix it before you launch.

Since then, releases have not been delayed anything like that first freeze. K3 dropped right at WAIC; Zhipu's Z.AI has been going from finished training to release very fast. Companies described weekly, sometimes daily, contact with regulators who are willing to impose costs but also want firms in their jurisdiction to succeed. A January Politburo study session had Xi talking about technological loss of control. A draft cybercrime law the same month would have required companies to monitor for and report bulk generation of malicious code — the kind of monitoring that, Labenz notes, might have mattered in the recent sandbox-escape incidents. He flags Dean Ball's reminder that Chinese draft law is lobbyable: an earlier accuracy mandate was walked back after firms explained that hallucinations cannot be promised away. In February, weeks after OpenClaw fever, the government warned about "relatively high security risks" in some versions. The state is watching the same trends everyone else is watching.

What would happen if a Chinese lab had an OpenAI- or Anthropic-style incident, with a model hacking third-party systems for days? He does not know, and he is careful not to recommend criminal charges against individuals in the American cases. He does note that if a human had done what those models reportedly did, it would be a felony, that Hugging Face involved law enforcement before knowing who was responsible, and that he suspects a harder slap in the Chinese system. The historical analogies he heard were Jack Ma being sidelined after criticizing financial regulation, the sudden tutoring crackdown, and tight limits on children's gaming — rules kids still evade with parents' phones. The phrase he came away with is that the Chinese government feels it can put the genie back in the bottle.

That confidence, he thinks, is why open weights look less terrifying from Beijing than from Washington. The Chinese internet does forget. Regulators are in contact with model developers, cloud providers, and inference shops, and they believe they can tell them to stop serving a model. Crypto is already banned, so the state has practice watching electricity. The State Grid had a major WAIC booth; it even helped a professor build a fake high-voltage training environment for robots. A basement hobbyist might slip through. A small illicit inference business with a 10x power bill, in his guess, would not. He does not think that assumption travels. Most other governments could not scrub a dangerous open-weight model off the internet, and he is not sure China is taking its spillover — especially bio risk that could blow back — seriously enough. Inside the border, though, they do not treat open weights as irreversible. On labor, the state is building its own monitoring and retraining programs, and one report said firms would not be allowed to fire people made redundant by AI. He doubts they can hold a line that expensive for long. They held COVID lockdowns longer than observers expected, so he is not certain. Either way, "we'll cede the race if we do anything" is, by the end of this rundown, a claim he considers easy to refute.

A standing complaint in the West is that Chinese firms which signed on in Seoul to publish frontier risk frameworks never did. Labenz does not excuse the broken promise. He does try to reconstruct the internal story. Western negotiators are often jaded — China will say it and not do it — and American labs are not pure on this either. Anthropic, he notes, once had a tighter responsible scaling policy with if-then commitments, then moved toward a "trust us" regime because it could not meet the old bar without, in its own telling, ceding the future. The Chinese version of the shrug is different. Safety standards, as companies seem to see it, are the government's job. They are in a registry, under weekly review, inside a division of labor they treat as legitimate. Writing their own public framework might even look like overstepping.

They also, in his reading, expect the standard to rise. Capabilities are going up; the 45-degree line is a society-wide cliché; hitting whatever the CAC asks is the job. He puts the capabilities gap at roughly nine months, a hotly debated number he treats as a reasonable center of credible estimates. Nine months ago Claude 4.5 Opus was the first time agents really worked; Chinese models are now getting there. When those labs are dealing with what OpenAI and Anthropic are dealing with now, he expects deployed safeguards to close a lot of the gap. Open weights may be the exception, because Beijing still thinks it can take a model offline at home. For now the startups doing the least on safety are still, in his summary, in catch-up mode: meet the government's number, assume it will get harder, and do not opine in public about what the number should be.

The largest gap he went looking for, he did not find. Is there alignment with Chinese characteristics — a Confucian constitution to sit beside Claude's? At the Temple of Confucius in Beijing, a Chinese model told him that Confucius's descendants, 79 generations on, still identify as such and perform rituals in his honor. If values could be projected through 79 recursively self-improved generations of AI, he said, we would be doing better than anyone should expect from a raw intelligence explosion. He asked around. He got no pointers. One professor called it an interesting idea and then said this may be the generation in Chinese history weakest on that tradition: "We're all engineers." Xi himself is a chemical engineer. The humanities are not where people went. Chinese safety work, as Labenz found it, sits on the OpenAI side of the durability-versus-character debate: clear rules, reliably followed. He did not find anyone trying to grow a model into a wisdom tradition. If a good future is several powerful AIs in some ecological balance, he thinks different traditions would be a feature. Right now that page is blank.

The episode's closing pitch is the one he started with. China does care. The research community is engaged and growing fast. Product companies still have work to do. The government is, in his judgment, pretty well on the ball, and both countries are trending in a better direction with a lot left undone. Whenever someone says China will never slow its companies down, this is the ammo. Part three will be more openly idealistic: what, if anything, the U.S. and China might do together.