GPT-5.2 Benchmarks: Why Scores Need Context

In this December 2025 review, I look beyond headline scores for GPT-5.2. It performs strongly on a number of tests, but the amount of reasoning compute spent on an answer matters when comparing models. The results and prices I discuss reflect the time of the video.

OpenAI reported that GPT-5.2 Thinking beat or tied expert professionals in 70.9% of comparisons on GDPval. That result concerns specified knowledge-work tasks across 44 occupations; it should not be read as proof that the model can replace an expert across an entire job.

The benchmark selects predominantly digital tasks and gives the model the necessary context. Real work often requires finding missing information and applying tacit knowledge. The evaluation also does not directly capture the consequences of a catastrophic mistake.

I tested a football spreadsheet task: gathering a club’s results and arranging them in an interaction matrix. The Pro model produced an impressive result, and I checked a selection of the scores. A lower-budget version gathered results but did not complete the matrix in my test.

I note that the release comparison did not include some contemporary competing models. A visual example shared by a Google representative showed a tighter motherboard segmentation by Gemini 3 Pro, but a single example is not a comprehensive comparison.

Test-time compute complicates leaderboards. A model can spend more tokens or money thinking about a question and often improve its result. Ideally, comparisons would show performance against cost or budget, rather than presenting a score without that context.

The ARC-AGI tests ask models to infer transformations from examples. I describe strong GPT-5.2 results and a large year-on-year efficiency improvement on ARC-AGI-1. On ARC-AGI-2, the displayed chart also shows scores varying with cost; different scaffolds can change a model’s result.

Benchmark selection is another source of ambiguity. SWE-bench Pro covers several programming languages and aims to reduce contamination, while other coding evaluations have different scopes. More reasoning tokens can improve scores, although models may also become more efficient per token.

Two evaluations of visual charts can produce different model rankings even though both appear to test chart understanding. That is a reminder to inspect the questions and measurement conditions before declaring one model universally better.

On Humanity’s Last Exam and GPQA Diamond, I compare reported results but also note possible benchmark contamination and answer-key errors. A published score is useful evidence about a test, not a complete measurement of scientific reasoning.

I also run my own SimpleBench, a private set of common-sense and spatial or temporal reasoning questions. In my runs, GPT-5.2 Pro scores 57.4% and Gemini 3 Pro 76.4%; I estimate a rough human baseline at 84%. These are my benchmark’s historical results, not a universal ranking.

Answers are compared programmatically against a private answer table. I observe that the base GPT-5.2 result in my test is lower than some earlier GPT results. Differences in benchmark construction and testing conditions still matter, and I do not have evidence that a provider optimized improperly for public tests.

My practical preference at the time is Claude Opus 4.5 for coding. I suggest choosing the model that works for the actual task, and demonstrate a model-comparison service I operate. Its product and pricing references are historical and should not be assumed current.

OpenAI reports strong performance on a four-needle long-context retrieval test. The chart shown in the video describes near-perfect performance through 256,000 tokens on that specific test; retrieving planted details is not the same as reliably reasoning over every long document.

The model improves modestly on OpenAI’s own pull-request evaluation, while a separate machine-learning engineering benchmark gives a mixed comparison with earlier models. I see progress rather than evidence of an imminent singularity.

I place the release in the context of OpenAI’s tenth anniversary and its leaders’ ambitions. Statements about future superintelligence are predictions. I also mention API prices and mental-health evaluations as they stood at release, not as current product advice.

To explain incremental progress, I imagine counting sheep across many fields: each sheep represents a human task. One approach counts them one by one, like models gradually improving across tasks; another would be a sudden breakthrough that counts all fields at once.

I speculate that even without a single dramatic breakthrough, repeated improvements could eventually cover many more tasks, including physical ones. The analogy is a way to think about progress, not evidence that every human task can or will be automated.