DeepScaleR: A Small Reasoning Model Tested on Competition Math

SUMMARY

In 30 seconds

In February 2025, Matthew Berman discusses Berkeley’s DeepScaleR-1.5B-Preview, a reinforcement-learning fine-tune of a distilled DeepSeek model. The researchers report gains on selected competition-math tests; Berman’s local demonstration produces a lengthy answer he suspects is incorrect.

1AIME comparisonThe team reports 43.1% pass@1 for DeepScaleR and lists 40.0% for o1-preview on AIME 2024.
2Outcome and process rewardsBerman contrasts feedback on final answers with feedback on intermediate steps and states his preference for the latter.
3Training resourcesThe reported run used about 3,800 A100 GPU-hours and cost $4,500, excluding prior model development.
4Local quantized testBerman runs one problem on an M2 Mac and doubts the generated answer.

Takeaways

DeepScaleR’s reported 43.1% AIME 2024 score exceeds the listed 40.0% o1-preview result under the cited comparison.
Selected math benchmarks do not establish superiority across all mathematics or general tasks.
The model starts from a previously distilled checkpoint and is further trained with reinforcement learning.
The reported $4,500 training run excludes earlier base-model and broader research costs.
Berman’s single quantized local test is not a reproduction of the full evaluation.
Long reasoning output and generation speed do not guarantee a correct answer.

TopicsReinforcement Learning Benchmarks Local inference Distillation Reasoning & test-time compute

FULL TEXT

Read the edited text by chapter

Show transcript

Transcript only

DeepScaleR and its benchmark

Berman introduces the small reinforcement-learning model and compares reported competition-math scores with o1-preview. The comparison does not establish general superiority.

In February 2025, Matthew Berman reviews DeepScaleR-1.5B-Preview, a Berkeley research model fine-tuned from DeepSeek-R1-Distill-Qwen-1.5B using reinforcement learning. He is excited by a small model exceeding OpenAI’s o1-preview on selected competition-math benchmarks. That result does not establish superiority on all mathematics, general knowledge or other tasks.

The researchers describe distributed reinforcement learning and longer-context training. Berman guesses that distributed might mean clusters spread around the world, but the term alone does not establish their physical locations. The team’s report says its work used 40,000 math problems and progressively scaled response lengths.

SLIDE 1 / 3Competition-math benchmark tableThe displayed table reports DeepScaleR and baseline results on specific math tests.

The team reports 43.1% pass@1 on AIME 2024 for DeepScaleR versus 40.0% for o1-preview, and 87.8% versus 81.4% on MATH-500 in its table. These are benchmark-specific historical results with particular evaluation settings, not proof that a 1.5B model exceeds o1-preview on every math problem or all versions of o1.

Rewards and training cost

He explains outcome and process rewards and discusses the researchers’ reported training run. The quoted cost does not include all prior model-development costs.

Berman contrasts an outcome reward, which assesses the final answer, with a process reward that gives feedback on intermediate steps. He prefers process feedback because it can identify where reasoning went wrong. That is his view; the video does not show an experiment establishing that process rewards are always better.

The research team argues that reinforcement learning can improve a small model already initialized with high-quality distilled reasoning data. Its reported AIME 2024 score rose from 28.8% for the starting model to 43.1% for DeepScaleR. Berman sees this as evidence that specialized small models may become useful for narrow tasks; it does not imply that the same improvement transfers to all domains.

SLIDE 2 / 3Research team’s training observationsThe excerpt discusses reinforcement learning on a distilled small model and iterative context-length scaling.

The researchers report approximately 3,800 A100 GPU-hours and $4,500 for their training run. This is the reported cost of their run, not the full cost of creating the base model, original distillation, data, research and infrastructure. Berman notes that the team released model weights and training code, enabling others to inspect and build on the work.

A local test

Berman downloads a quantized version and runs one AIME problem on an M2 Mac. He doubts its answer despite its lengthy output.

Berman downloads a quantized GGUF version and runs it locally in LM Studio on an M2 Mac while recording and performing another training task. He estimates the displayed quantized file at about 1.12 GB, whereas the full-precision file is larger. Hardware memory, context length and runtime affect whether a particular device can run it; a phone demonstration is not shown.

He gives the model an AIME problem. It produces a lengthy response and a final answer that Berman suspects is wrong; the screen shows approximately 21,618 generated tokens at 44.7 tokens per second in this run. A long explanation or fast generation does not establish correctness, and one quantized local test cannot reproduce the full benchmark.

SLIDE 3 / 3Local AIME responseThe quantized model’s lengthy output ends with an answer that Berman questions.

Berman concludes that small, locally run models can be useful for specialized reasoning. He is an investor in LM Studio, as disclosed in the video description. The broader comparison remains limited to the researchers’ reported tests and the single local demonstration.