Open-Weight AI: Theo Examines NVIDIA’s Letter and Anthropic’s Response

NVIDIA CEO Jensen Huang shared an open letter in support of open-weight AI models, signed by many technology companies including OpenAI. Anthropic did not sign. I read the letter and Anthropic CEO Dario Amodei’s response to understand where the participants agree and disagree.

The immediate context includes a US official’s allegation that Moonshot used Anthropic model outputs in large-scale distillation for Kimi K3 and obtained access to advanced NVIDIA hardware. The allegation is disputed in the discussion and should not be treated as established solely because it was made in a post.

NVIDIA has a commercial stake in the debate: both training and serving models can require chips, and wider model adoption can mean more hardware demand. That incentive does not by itself determine whether the arguments in the letter are right or wrong.

Critics of the official’s statement question how the alleged distillation happened and point to broader questions about AI training data. I am skeptical of the allegation and of treating all distillation as theft. My skepticism, however, is not proof of what Moonshot did or did not do.

The letter, titled Open Weights and American AI Leadership, argues that the world needs both open- and closed-model approaches. Its signatories ask policymakers to support an ecosystem in which more organizations can build, adapt and deploy models. NVIDIA’s own description confirms that it joined the letter. [NVIDIA discussion](https://blogs.nvidia.com/blog/open-world-models-physical-ai/).

The authors draw a parallel with open-source software: shared software foundations let organizations reuse and improve work rather than each building everything alone. I agree that this can give small businesses and nonprofits access to capabilities they otherwise could not afford.

The letter defines open weights in terms of downloading, inspecting, modifying and running models on one’s own infrastructure. Open weights and fully open-source model development are not identical: access to training data, code, methods and licenses varies between releases.

The proposed benefits include choosing an appropriately sized model for a task, avoiding a single provider and controlling where data is processed. Running a model yourself still requires hardware, operations and an applicable license; the weights being available does not make deployment costless.

The letter also argues that openness can increase competition across model developers, cloud providers, chips and applications. I find the ability to keep data in infrastructure under an organization’s control compelling, particularly when a task does not require the most expensive hosted model.

The signatories acknowledge a distinct risk: once weights are released, copies and modifications are difficult to recall or monitor. A hosted service can change access controls after a problem is found; a released file may remain available. This is a genuine safety trade-off, not something to wave away.

The letter’s answer is that defenders also need strong tools, and that broader access allows more teams to test models and fix weaknesses. I previously liked the idea of giving infrastructure maintainers early, restricted access for defensive work. The question is whether those programs reach enough people quickly enough.

I cite the OpenAI–Hugging Face security incident as a reason to worry about the gap between attackers and defenders. My video speculates about the identity of the model and the exact sequence. Later public investigations describe the incident in more detail; they do not establish every claim I make here. [OpenAI’s later account](https://openai.com/index/hugging-face-incident-and-the-road-ahead/).

The letter argues that closed models are not inherently safe either: access can be concentrated, failures can be difficult for outsiders to inspect, and providers can themselves be compromised. It presents broad testing and independent scrutiny as potential safety benefits. Whether those benefits outweigh misuse risks is an empirical and policy question.

The policy recommendations include compute access for researchers and startups, shared datasets and evaluation tools, and avoiding premature restrictions that suppress competition. Those measures could support a wider ecosystem, while also benefiting NVIDIA as a hardware supplier. Both observations can be true.

On distillation, the letter distinguishes using one model’s outputs to improve another from unlawful extraction. It calls for targeted legal and commercial responses to specific misconduct rather than a sweeping restriction on the technique. What is permitted depends on the facts, terms and applicable law.

I support the letter’s emphasis on openness and competition, though I recognize the signatories’ incentives. I am concerned that overly broad restrictions could push development elsewhere and make it harder for organizations to choose the tools that fit their needs.

Amodei’s response rejects a category-wide ban on open-weight models. He calls models without dangerous capabilities a public good and says protectionist bans on US business use would not address his main concerns. That is his stated position, even though I disagree with other parts of it. [Anthropic’s response](https://www.anthropic.com/news/position-open-weights-models).

His first concern is that authoritarian governments could develop leading systems for military dominance or repression, whether or not those systems are open-weight. I understand the concern but question how he weighs different ways competitors can catch up. His national-security assessment remains an argument, not a demonstrated outcome.

His second concern involves misuse in cyber or biological attacks and alignment failures. He argues that released weights can be harder to monitor or withdraw. He also says a ban on use by legitimate US businesses would not by itself stop malicious actors.

Amodei proposes restricting advanced chips and chipmaking equipment to China, and combating circumvention. This is his proposed policy and rationale; its effectiveness and consequences are contested questions. I also question his emphasis on large-scale distillation as an explanation for competitors’ progress.

He calls for action against industrial-scale distillation operations, arguing that they can help competitors get more capability from limited compute. I disagree with the emphasis and see distillation as a useful and widespread technique. Neither model rankings nor our disagreement independently prove or disprove a particular allegation.

I suggest retaining one’s own model interaction logs for future work. Such logs can contain private code, credentials or other sensitive information, and any later use is subject to applicable terms and law. The practical choice is to know what is being stored and protect it appropriately.

Amodei’s third proposal is mandatory pre-release safety testing for sufficiently capable models, whether open or closed. He argues the risks should be tested rather than presumed and says effective testing would need international participation. I consider testing a more constructive direction than a blanket prohibition.

Amodei says he agrees with the letter on access, competition and customer control, and on targeted treatment of distillation concerns. He disputes the assumption that open access necessarily makes defense stronger than attack. In particular he worries about an imbalance in biological risk, which he says should be evaluated empirically.

I remain frustrated by Anthropic’s language around industrial distillation. I think the technique deserves to be separated from particular unauthorized conduct, and I do not find the available claims sufficient to explain all competitors’ progress. That is my interpretation rather than a finding about any one company.

My conclusion is that open-weight models provide important choices, while meaningful safety tests and misuse risks deserve attention. I like using Anthropic’s model and still disagree with the company’s policy emphasis. Readers can examine the letter and response and make their own judgment.

The debate is not a simple choice between supporting every open release and banning every model. It involves access, incentives, security, evidence and how narrowly rules can address a demonstrated problem. I am interested in how developers and users weigh those trade-offs.