30 秒でわかる
Karpathy discusses the challenges of specialization, capability retention and the cost of deeper model customization.
- 1Capability retentionFine-tuning can risk existing abilities.
- 2ContextKarpathy calls context a relatively cheap customization mechanism.
- 3WeightsChanging weights can affect the whole model.
ポイント
- Context is a relatively cheap way to adapt behavior.
- Fine-tuning changes model weights.
- Specialization must preserve capability and justify its cost.
チャプターごとに整えた全文を読む
Adapting models
Andrej Karpathy argues that the methods for deeply adapting AI models are still developing. He gives fine-tuning without loss of existing capabilities as one challenge.
Andrej Karpathy argues that the methods for deeply adapting AI models are still developing. He gives fine-tuning without loss of existing capabilities as one challenge.
He contrasts context windows, which he describes as a relatively cheap and useful way to customize behavior, with changing the model itself.
Weights and specialization
Karpathy says continual learning and specialization require better ways to adjust model weights and improve performance in a particular area.
Karpathy says continual learning and specialization require better ways to adjust model weights and improve performance in a particular area.
Changing weights can affect the whole model and its capabilities, making it trickier than changing context. The speakers add that such customization must also be economical.
同じチャンネルの記事
38:06EN
No Priors: AI, Machine Learning, Tech, & Startups· 2026年10月29日ポッドキャスト · 3 人Serval’s Jake Stauch on coding agents for enterprise ITゲスト Jake Stauch · ホスト Sarah Guo, Elad GilStauch explains why Serval is challenging ServiceNow, how it approaches permissions and approvals, and why background agents remain difficult to deploy safely.
1:07ENNo Priors: AI, Machine Learning, Tech, & Startups· 2026年10月27日ポッドキャストGeorge Kurtz on How AI May Change Cyberattacksゲスト George KurtzThe CrowdStrike CEO describes his assessment of evolving threats.
39:23ENNo Priors: AI, Machine Learning, Tech, & Startups· 2026年10月15日ポッドキャスト · 2 人The Defenders' Window: Greg Brockman on AI-Generated Knowledge, Drug Discovery, and the New Physics of Cybersecurityホスト Sarah Guo, Elad GilOpenAI's president explains why solving open math problems is a proof point for science, how proactive agents change product form, and why security spending must exceed software spending.
1:04ENNo Priors: AI, Machine Learning, Tech, & Startups· 2026年10月14日ポッドキャストWalter Goodwin on the Slow Steps in Chip Designゲスト Walter GoodwinA chip founder explains why automation does not remove every computational bottleneck.
1:09:44ENNo Priors: AI, Machine Learning, Tech, & Startups· 2026年10月9日ポッドキャスト · 2 人10,000 GB300s for Four Weeks: Reflection’s Open-Model Ledgerホスト Sarah Guo, Elad GilMisha Laskin tells No Priors how Reflection trained Beam, why reinforcement learning outspent pre-training, and why the West needs its own open weights.
0:39ENNo Priors: AI, Machine Learning, Tech, & Startups· 2026年10月8日ポッドキャストMichael Lee on Rethinking Organizations with AIゲスト Michael LeeA business leader argues that the opportunity goes beyond faster tasks.