Apple’s “Illusion of Thinking”: Puzzle Limits and a Code-Assisted Test

Apple’s paper “The Illusion of Thinking” has sparked a debate about what reasoning models actually do. I review its experiments, its concerns about benchmark contamination and generalization, and its findings about how model performance changes with puzzle complexity.

The paper studies models that generate extended reasoning traces before answering, including Claude and DeepSeek variants. The researchers question how much benchmark progress reflects reasoning, extra inference compute or possible exposure to benchmark material. My description of contamination as models “cheating” is a concern about possible exposure, not proof that a particular model was intentionally trained on a particular test.

Established mathematics and coding benchmarks often report final-answer accuracy, which makes it difficult to vary complexity systematically or inspect intermediate attempts. The paper proposes controllable puzzles with consistent rules and simulator-based checks, so researchers can examine both solutions and reasoning traces.

The four environments are Tower of Hanoi, checker jumping, river crossing and Blocks World. In Tower of Hanoi, discs must move between pegs one at a time without placing a larger disc on a smaller one. Adding discs, pieces, entities or blocks changes problem complexity while keeping the basic rules.

The authors argue these constructed environments help address contamination concerns and allow exact verification. Their questions include whether models generalize, how performance scales with complexity and how reasoning models compare with standard models given equivalent inference budgets.

The paper discusses prior comparisons on MATH-500 and AIME 2024, then reports its own comparisons on MATH-500, AIME 2024 and AIME 2025. The authors say the performance gap is larger on the newer AIME tests and consider both genuine reasoning advantages and differences in contamination as possible explanations; they do not establish which caused the gap.

One comparison uses pass@k, examining whether at least one of multiple generated candidates is correct under a compute budget. I describe this as choosing the best solution, but the metric is an upper-bound evaluation: identifying the correct candidate in practice would require a suitable verifier or selection method.

The puzzle plots compare Claude 3.7 Sonnet with and without thinking and DeepSeek-R1 with DeepSeek-V3. In the experiments, easy instances are often solved by both; thinking models have an advantage at intermediate complexity; at sufficiently high complexity both can fall to zero accuracy. The threshold differs across puzzles and models.

The researchers generate 25 samples per puzzle instance and report average performance. They also compare performance at similar inference-token budgets. The three regimes are a result of these selected model pairs, tasks and evaluation conditions, rather than a claim that every model fails every hard problem.

The paper reports that reasoning-token use initially rises with puzzle complexity but falls after a point as performance collapses, despite an available token budget. I wonder whether that resembles “giving up,” but the token pattern alone does not establish the model’s motivation.

The authors examine where correct and incorrect candidate solutions appear in reasoning traces. On some easier instances they find correct solutions early followed by further exploration, which they call overthinking; on moderately difficult problems correct solutions more often appear later. The patterns vary among the puzzle environments.

In the Tower of Hanoi tests, the researchers also provide an explicit algorithm in the prompt. They report that this did not prevent the observed failure around the same complexity point, suggesting difficulty executing and verifying long sequences in that setup. The first error appeared at different points across puzzle types.

I contrast the debate with a clip of Ilya Sutskever’s commencement remarks. He predicts AI will eventually perform anything people can learn, arguing that the human brain is a biological computer and asking why a digital computer could not do the same. This is his broader prediction, not a result of Apple’s puzzle experiments.

The authors acknowledge that their experiments cover a narrow slice of reasoning and may not reflect diverse real-world or knowledge-intensive tasks. I agree that poor results on these puzzles alone do not prove models cannot think: models also show abilities in coding and other domains.

My biggest question is whether a model should get credit for writing code to solve the puzzles. I ask Claude 3.7 to build a Tower of Hanoi simulation in HTML and JavaScript, with up to ten discs and a solve button, without providing the algorithm in the prompt.

The generated program reaches the solved state for ten discs in 1,023 moves. I then request a 20-disc version; the video shows it running, with a visual formatting issue, and displays a minimum of 1,048,575 moves. The clip does not show the 20-disc run finishing.

A program can execute the algorithm after the model writes it, which is a different setting from having a model directly enumerate moves in natural language. I ask whether that tool-assisted route should count as solving the problem. The demonstration illustrates an alternative workflow rather than directly replicating the paper’s evaluation.