DeepSeek V3.2: Reasoning, Sparse Attention and Agent Training

요약

30초 요약

Matthew Berman discusses DeepSeek V3.2 and the higher-compute V3.2-Speciale, drawing on the developer’s published results and technical report. He examines benchmark scores, sparse attention, reinforcement learning and synthetic agent tasks, while noting that tool-use performance remains below the leading proprietary models.

1Two model variantsDeepSeek announced V3.2 and the higher-compute V3.2-Speciale; its release material described Speciale as API-only at launch.
2Reasoning benchmarksDeepSeek’s table reports 96.0 on AIME 2025 for Speciale, compared with 93.1 for V3.2 Thinking, with higher token use.
3Sparse attentionDeepSeek says its attention mechanism reduces computational complexity while preserving long-context performance.
4Post-training and agent tasksThe technical report describes a large post-training budget and synthetic environments for reinforcement learning.
5Model sizeBerman cites 671 billion total parameters and 37 billion active parameters in the mixture-of-experts model.

핵심 정리

DeepSeek reported strong reasoning results for V3.2-Speciale, with greater reasoning-token usage than the standard thinking model.
DeepSeek Sparse Attention aims to reduce attention complexity for long contexts.
The team reports spending more than 10% of the pre-training compute budget on post-training.
Synthetic environments and prompts were used to train agentic tool use.
The 671-billion-parameter mixture-of-experts model activates 37 billion parameters per token, while its full weights still demand substantial memory.

주제Foundation Models Open-weight models Reasoning & test-time compute 에이전트 엔지니어링 모델과 학습

전문

챕터별로 다듬은 전문 읽기

원본 자막과 대조

원본 자막만 보기

The V3.2 release and benchmarks

Berman introduces the two variants and compares reported reasoning, coding and science benchmark scores. The results come from DeepSeek’s own published table.

DeepSeek V3.2 has arrived, and I think it marks another important moment for open models. DeepSeek reports that its higher-compute model achieved gold-medal-level performance on the 2025 International Mathematical Olympiad and International Olympiad in Informatics. These are developer-reported evaluation results; they do not establish that the model leads every closed model on every task.

슬라이드 1 / 5DeepSeek’s V3.2 launch announcementThe launch post distinguishes the standard model from the higher-compute Speciale variant and shows selected benchmark comparisons.

DeepSeek announced V3.2 and V3.2-Speciale as reasoning-first models. Its launch post presents V3.2 as the successor to V3.2-Exp, available through the app, web and API, and describes Speciale as a higher-compute variant available through the API at launch. The post also says Speciale did not support tool use at that point.

The company’s benchmark table compares the models with GPT-5 High, Gemini 3.0 Pro and Kimi K2 Thinking; I note that Claude Opus 4.5 is absent. On AIME 2025, the table reports 94.6 for GPT-5 High, 95.0 for Gemini 3.0 Pro, 93.1 for V3.2 Thinking and 96.0 for Speciale.

슬라이드 2 / 5Reasoning and coding benchmark tableDeepSeek’s reported scores also show the reasoning-token counts used for each model.

The numbers in parentheses are the reasoning tokens used. DeepSeek’s standard thinking model uses fewer tokens than Speciale on the examples shown, while Speciale spends substantially more to achieve higher scores. That extra inference compute matters when comparing both performance and cost.

On LiveCodeBench, the displayed results are 84.5 for GPT-5 High, 83.3 for V3.2 Thinking and 88.7 for Speciale. On GPQA Diamond, the table shows 85.7 for both GPT-5 High and Speciale, and 91.9 for Gemini 3.0 Pro. These are selected benchmarks, not a guarantee of the same ordering in every real task.

Attention, reinforcement learning and agent data

The technical report describes sparse attention, scaled post-training and synthetic agentic tasks. Berman explains the intended benefits.

DeepSeek’s report describes several technical changes. The first is DeepSeek Sparse Attention, or DSA, which the authors say substantially reduces attention’s computational complexity while preserving performance in long-context scenarios. More efficient attention can make longer contexts less expensive to process.

슬라이드 3 / 5DeepSeek V3.2 technical report abstractThe report summarizes sparse attention, reinforcement learning and large-scale agentic task synthesis.

Conventional full attention becomes costly as the sequence length grows because its attention computation scales quadratically. DeepSeek describes reducing the core attention complexity from O(L²) to O(L × K), where L is sequence length and K is the number of selected tokens. The savings depend on the value of K and do not imply that every part of model inference becomes linear.

The second component is a scalable reinforcement-learning framework and more post-training compute. DeepSeek says V3.2 performs comparably to GPT-5 on its evaluated reasoning tasks and reports that Speciale reaches gold-medal-level results on the 2025 IMO and IOI. Those comparisons reflect the developer’s evaluation conditions.

Third, DeepSeek built a large-scale pipeline to synthesize agentic tasks. It generated training scenarios intended to combine reasoning with tool use, producing data for reinforcement learning rather than relying entirely on manually created examples. The aim is better generalization and instruction following in interactive environments.

Training details and practical deployment

Berman returns to the compute budget and synthetic environments, then discusses tool-use limits and the memory required to run the model.

DeepSeek’s report says its post-training compute budget exceeded 10% of the pre-training cost. That is a comparison with pre-training compute, not a statement that reinforcement learning used 10% of the entire project budget. The company also reports generating more than 1,800 distinct environments and 85,000 complex prompts for agentic tasks.

슬라이드 4 / 5Post-training budget and synthetic tasksThe report describes post-training compute above 10% of pre-training cost and more than 1,800 environments.

The advantage of sparse attention is the difference between quadratic growth in the main attention operation and growth proportional to L × K. Berman describes the practical goal as controlling the cost of longer sequences, while the actual runtime and expense depend on implementation, hardware and other model operations.

DeepSeek reports that V3.2 narrows the gap between open and closed models on tool-use benchmarks, while still trailing frontier systems in that evaluation. The displayed paper also notes that redundant self-verification can generate long trajectories and exceed the 128K context limit in some tasks.

슬라이드 5 / 5Tool-use evaluation and limitationsThe displayed report describes the remaining gap with frontier models and long-trajectory context limits.

The model has 671 billion total parameters and activates 37 billion at inference in its mixture-of-experts architecture. I estimate roughly 700 GB of VRAM for FP8 weights and 1.3 TB for BF16, illustrating that a relatively small active parameter count does not eliminate the memory needed to hold the full model. The release includes open weights under an MIT license; the Speciale variant’s availability at launch was more limited.