The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
TL;DR
By testing language models on increasingly complex puzzles, the authors find that models that “think” before answering have an advantage at intermediate difficulty but, like non-thinking models, eventually fail on the hardest cases tested.
- The authors varied the size of four puzzles—Tower of Hanoi, Checker Jumping, River Crossing, and Blocks World—and used simulators to check proposed moves and final states, rather than judging answers alone.
- They report 25 samples per model and puzzle instance, allow up to 64,000 generated tokens for Claude and DeepSeek models, and filter responses for the requested format.
- At comparable inference-token budgets, non-thinking models were more efficient and sometimes more accurate on simple puzzles, thinking models performed better on moderately difficult ones, and both fell to near-zero accuracy on the hardest cases.
- Thinking-token use generally rose with difficulty but often declined near the point of accuracy collapse despite unused generation capacity, which the authors interpret as a possible limit on scaling reasoning effort.
- In Claude’s analyzed thinking traces, correct attempts often appeared early on simple puzzles and later on moderately difficult ones, while on harder puzzles the model found no correct attempt and often fixated on an early error.
- Some models achieved over 50% accuracy on Tower of Hanoi instances requiring about 100 moves but consistently failed River Crossing instances requiring about 11; the authors suggest differences in familiarity with training examples as one possible explanation.
- Supplying solution procedures did not substantially delay failure on Tower of Hanoi or Checker Jumping, although the printed Checker Jumping pseudocode does not return successful results from its loop, complicating interpretation of that test.
- These findings concern unaided, precisely checkable puzzles, and the authors caution that they may not transfer to open-ended or knowledge-intensive reasoning; black-box access also limits insight into the models’ internal mechanisms.
The questions
01What problem or question does the paper address?
The paper asks how reliably large reasoning models, which generate extended thinking before answering, solve problems as complexity increases and how they compare with models that do not use extended thinking.
The authors identify a gap in existing evaluations: mathematical and coding benchmarks often focus on final-answer accuracy, may overlap with models’ training data, and reveal little about the quality of intermediate reasoning. Consequently, it can be difficult to tell whether improved scores reflect better reasoning, greater exposure to relevant data, or more computation used while answering.
To investigate these questions, the authors build a testbed of four puzzles whose size can be varied while their basic rules remain consistent, using computer simulators to check proposed moves and whether they reach the goal. The puzzles involve moving disks between pegs, swapping colored checkers, transporting people safely across a river, and rearranging stacks of blocks. The authors primarily track how models respond to increasing complexity within each puzzle, rather than assuming that different puzzles are equally difficult for a model.
They compare thinking and non-thinking models using accuracy and the chance that at least one attempt succeeds within an equivalent token budget, where tokens are pieces of generated text. They also measure reasoning effort by counting thinking tokens: a higher count means the model generated a longer thinking trace. These comparisons test when extended thinking helps and whether models continue to use more thinking tokens as problems become harder.
The prompts ask models to include complete move lists while exploring solutions, allowing the authors to extract and check candidate solutions from their written thinking. The authors examine when correct and incorrect candidates appear and whether models recover from earlier mistakes as complexity increases.
For two puzzles, the authors also provide a step-by-step solving procedure to test whether difficulty lies only in discovering a strategy or also in executing one reliably. They examine where the first invalid move occurs to study how failures vary across puzzle sizes and types.
In these tests, the authors report that non-thinking models are competitive or better on easy puzzles, thinking models gain an advantage at moderate complexity, and both approaches fall to near-zero accuracy on sufficiently complex puzzles. They also report that thinking-token use often falls near the point of failure and that providing procedures does not prevent failure in the two puzzles tested. The paper does not establish that these patterns apply to all reasoning tasks: the authors note that their structured puzzles cover a narrow range of tasks and that mostly black-box access limits analysis of models’ internal mechanisms.
02Why does it matter?
Understanding the capabilities and limits of reasoning models matters because it can guide researchers trying to improve them and inform how the systems are designed and deployed.
Reasoning models are language models designed to produce a thinking process before giving an answer, and their performance has raised hopes for more generally capable artificial intelligence. The authors argue that it remains unclear how much of their performance comes from specialized training, more computation spent producing answers, or exposure to similar examples during training.
Existing tests can be affected by data contamination, meaning that test problems or similar material may already have appeared in the model’s training data. That makes it harder to tell whether good performance reflects a broadly useful problem-solving ability or familiarity with the test. Tests that emphasize final answers also reveal little about the structure and quality of the steps a model took to reach them.
To investigate these questions, the authors use puzzles whose difficulty can be adjusted while their basic rules stay consistent, and whose proposed solutions can be checked by computer programs. This lets researchers compare difficulties and inspect attempted solutions in a model’s written thinking, rather than looking only at its final answer. The authors want to assess the model’s own problem-solving and execution, not just whether it can obtain a correct answer using an external tool.
These tests cannot establish the full range of a model’s reasoning abilities: puzzles cover only a narrow slice of reasoning, and precise step-by-step checking may not transfer to open-ended problems. The authors also note that most experiments rely on external access to closed models, which limits their ability to examine how those models work internally.
03What makes it difficult to address?
Determining how well a language model can reason is difficult because correct answers can have several explanations, and the model’s problem-solving process is hard to inspect and test across different kinds of tasks.
The paper studies reasoning models, language models designed to generate an extended thinking process before answering, and asks whether their problem-solving abilities generalize as tasks become more complex.
Many established math and coding benchmarks, or standard tests, emphasize final-answer accuracy and may suffer from data contamination, meaning that test material may have appeared in a model’s training data. A better score alone therefore cannot show whether the improvement comes from training exposure, extra computation spent generating thinking, or reasoning abilities developed during training.
To examine more than final answers, the authors use puzzles whose sizes can be changed while their basic logic stays the same, and they check proposed solutions with simulators: programs that track puzzle states and validate moves against the rules. These puzzles still require models to track changing states, follow constraints, and coordinate multiple steps; in two puzzles, even giving models a solving algorithm did not prevent failure at roughly the same difficulty.
The number of moves required does not by itself show how difficult different puzzles will be for a model. For example, one tested reasoning model had near-perfect accuracy on a 31-move Tower of Hanoi puzzle, which involves moving disks between pegs, but failed an 11-move River Crossing puzzle, which involves moving people safely across a river. The authors suggest that differences in familiarity from training may help explain this disparity, but they do not establish that as its cause.
Even when researchers can inspect a model’s generated thinking text, most of the experiments rely on black-box access, which does not allow direct examination of the model’s internal states or architecture. The authors also caution that puzzles cover only a narrow range of reasoning, and the precise checking possible in them may not work for open-ended or knowledge-intensive problems.
04What does existing work leave unresolved?
Earlier work had not established how reliably reasoning models generalize as problems become more complex, what causes their improvements, or how their intermediate reasoning changes with difficulty.
Reasoning models are language models that produce extended “thinking” before answering, and earlier studies had shown that generating intermediate reasoning and checking it can improve performance. The authors say it remained unclear how much of their improvement came from exposure to mathematical data, extra computation spent generating thoughts, or training intended to improve reasoning.
Existing evaluations mainly used established mathematics and coding tests and emphasized whether the final answer was correct. The authors argue that these tests may include material seen during training and do not allow complexity to be varied under controlled conditions, making familiarity difficult to distinguish from reasoning ability. They also provide little insight into the structure and quality of the steps leading to an answer.
Earlier studies compared reasoning models with standard models without extended thinking, allowing multiple attempts under the same budget for generated text. They found that standard models could eventually reach comparable performance on certain mathematics benchmarks, but those comparisons did not settle how the advantage of thinking changes as problem complexity rises.
Earlier researchers had also reported “overthinking,” in which a model continues producing redundant reasoning after finding a solution, but its relationship to problem complexity remained insufficiently understood. Controlled puzzle environments were not entirely new, and earlier studies had found improvements by reasoning models on puzzles and decision-making tasks, but the authors identify a need for deeper analysis of their intermediate reasoning.
05What is the core intuition behind the approach or argument?
The paper’s core idea is to test models that write a thinking process before answering with puzzles that get harder while keeping the basic rules fixed, so their proposed moves can be checked step by step rather than judging only their final answers.
For example, in Tower of Hanoi, the goal is to move a stack of disks between pegs without placing a larger disk on a smaller one. Adding disks makes the shortest solution longer while preserving those rules, giving the authors a way to study what changes as a task requires more steps. The authors request lists of moves and check them with simulators: programs that track the puzzle’s state, verify each move, and check whether the goal is reached.
The authors compare reasoning models with standard models that do not produce an extended written thinking process, using comparable budgets of generated tokens, or pieces of text. Across the tested puzzles, standard models were often as accurate or more accurate on easy problems, reasoning models gained an advantage at moderate difficulty, and both reached near-zero accuracy on the hardest problems tested.
The amount of written thinking is another clue: the authors use the number of thinking tokens as a measure of reasoning effort, so a higher count means more thinking text was generated. That count generally rose as puzzles grew harder but then often fell near the point where accuracy collapsed, even though the models usually had generation budget left.
Checking candidate solutions inside one model’s written thinking also helps show what the extra text contains: on simple puzzles it often produced a correct sequence early and then explored incorrect ones, while on moderately harder puzzles correct sequences tended to appear later. At higher complexities, the authors found no correct candidate solutions in those traces and observed the model fixating on early errors.
In two puzzles, supplying a solving procedure still left the models failing at roughly the same difficulty, which the authors take as evidence of difficulty executing and checking steps, not just discovering a strategy.
These findings concern a narrow set of structured puzzles requiring explicit move lists; they do not establish how well the models reason in every real-world setting. The authors also note that familiarity with particular puzzles may affect performance and that their largely black-box access limits analysis of the models’ internal mechanisms.
06What does the paper find, prove, or provide?
The paper finds that tested AI language models that generate an extended thinking process can outperform models without that process on moderately difficult puzzles, but both types eventually fail on harder tested puzzles, and it provides a puzzle-based framework for studying those failures.
The framework uses four adjustable puzzles involving moving disks, jumping counters, crossing a river, and rearranging blocks, and checks proposed moves and final states with simulators: programs that enforce each puzzle’s rules. The paper provides descriptions of the puzzles, prompts, simulators, and a method for extracting attempted solutions from the models’ written thinking.
In comparisons using equal budgets for generated text, models without extended thinking were comparable or better and more efficient on easy puzzles, while thinking models gained an advantage on moderately difficult ones. On harder puzzles, accuracy—the share of attempts that completely solved a puzzle—fell to near zero for both types, although thinking models generally reached that point at higher difficulty.
In one model’s written thinking, the authors found that correct solutions often appeared early on easy puzzles, after which the model continued exploring incorrect alternatives. On moderately difficult puzzles, correct solutions tended to appear later, while on the hardest puzzles the model often failed to find one and became stuck on early errors.
The amount of generated thinking text generally increased as puzzles became harder, but often decreased near the point where accuracy collapsed, even though the models usually had room to generate more text. The authors interpret this pattern as a limit on how the tested models increase their thinking effort as problems become more complex.
Giving models solving procedures for two puzzles did not reliably prevent the collapse, which the authors take as evidence of difficulty carrying out steps as well as discovering a strategy. Models also produced much longer valid sequences in some puzzles than in others and sometimes made an error earlier on a larger instance of the same puzzle. The authors suggest that differences in familiarity from training may help explain variation between puzzles, but the paper does not establish that cause.
The findings concern a narrow set of puzzles, and the authors filtered samples to retain responses that followed the requested format. The paper also notes its limited access to models’ internal workings, so its results do not prove a universal limit on reasoning or establish exactly why the failures occurred.
07How well do the evidence or proofs support the claims?
The evidence supports the reported puzzle failures and an advantage from extended thinking at some difficulty levels, but it is less conclusive about why the failures occur or whether they reveal a general limit on reasoning.
The paper studies reasoning models, which are language models designed to produce a thinking process before answering, alongside counterparts that do not use extended thinking. The authors vary the sizes of four puzzles involving disks, checkers, river crossings, and blocks, and use programs called simulators to check whether proposed moves are legal and reach the goal. This design gives stronger evidence about step-by-step puzzle performance than a final-answer check alone, but the authors acknowledge that such puzzles cover only a narrow range of reasoning tasks.
Across the tested reasoning models, accuracy—the share of samples producing a correct complete solution—generally fell toward zero as puzzle size increased, with the failure point depending on the model and puzzle. That supports a failure to solve sufficiently difficult instances under the tested conditions, not a proof that these models lack general reasoning ability in every setting.
For each puzzle instance and complexity level, the authors report 25 samples per model and filter samples for the requested response format. The reported setup does not quantify how filtering affected the results, so its effect on the conclusions about all model responses remains unclear.
The authors’ comparison of thinking and non-thinking models uses pass@k, the chance that at least one of k generated answers is correct, under equivalent generated-token budgets; a higher value means a better chance of success. For Claude 3.7 Sonnet with and without thinking, and for DeepSeek-R1 versus DeepSeek-V3, the puzzle results averaged across environments show non-thinking models doing comparably or better with fewer tokens on easy problems, thinking models gaining an advantage at medium difficulty, and both failing at high difficulty. An additional Qwen model pair showed similar trends, but these comparisons establish the three regimes only for the models, puzzles, and budgets tested.
The authors measure reasoning effort by the number of thinking tokens generated, with a token being a piece of text; more tokens mean a longer thinking output, not necessarily better reasoning. In the tested models, thinking-token use generally rose with puzzle complexity and then fell near the accuracy collapse, even though most runs were below their generation limits; the decline was more pronounced in the o3-mini settings than in Claude 3.7 Sonnet with thinking. This supports the observed pattern of token use, but, in my assessment, it does not prove that a different prompt or method of allocating computation could never help.
For a closer look at how solutions developed, the authors analyzed Claude 3.7 Sonnet’s thinking traces—the text it generated while working—and checked candidate move lists with the puzzle simulators. On easier problems, correct candidates often appeared early and were followed by incorrect exploration; at moderate difficulty, correct candidates tended to appear later, while at high difficulty the analysis found no correct candidates. These observations support the authors’ descriptions of wasted exploration and limited self-correction in the extracted traces, but they do not expose the models’ internal states or establish why those patterns occurred.
The trace analysis also has a format limitation: the prompts required complete candidate move lists, and the extraction process retained only unique move sequences. In my assessment, the analysis therefore characterizes the candidates it could extract rather than every way the model might have worked on a puzzle.
The authors argue that failure is not merely a matter of running out of context space: in Tower of Hanoi, collapse began at sizes whose required move lists fit within the stated limits, and the first invalid move often appeared well before the end of a solution. That is meaningful evidence against a simple context-limit explanation for those failures, although it does not rule out other difficulties associated with producing long, exact sequences.
To test whether random selection of generated text caused the collapse, the authors also ran DeepSeek-R1 with greedy generation, which selects the most likely next token instead of sampling; collapse still occurred across the puzzles. This weakens an explanation based only on random sampling, but, in my assessment, it does not by itself identify the cause of the errors or eliminate the possibility that mistakes accumulate over a sequence.
When the authors supplied solution algorithms for Tower of Hanoi and Checker Jumping, reasoning models still collapsed at roughly the same puzzle sizes; a further DeepSeek comparison found modest gains at some sizes but unchanged collapse points for thinking and non-thinking models. This supports a difficulty with executing exact steps, not just discovering a strategy, in those experiments. However, as printed, the Checker Jumping pseudocode does not pass the recursive result back from its main loop, and its displayed checks do not verify the jumped checker, although the accompanying note calls for backtracking and checking jumps. In my assessment, that makes the Checker Jumping result less decisive as a test of following a fully specified correct algorithm.
The authors also report inconsistent failure positions: for Claude 3.7 Sonnet with thinking, a larger Tower of Hanoi instance could fail before 50 moves even though a smaller one proceeded beyond 100. The same model could solve a five-disk Tower of Hanoi nearly perfectly, yet fail on a three-pair River Crossing puzzle despite its shorter solution. These examples support the claim that solution length alone does not predict performance across puzzles, but the authors’ suggestion that training-data familiarity explains the difference remains unestablished because the relevant training exposure is not known.
The math-benchmark comparison also leaves its explanation open: thinking and non-thinking models performed similarly on MATH-500 under matched token budgets, but the gap widened on AIME24 and AIME25. The authors say that either problem complexity or differences in exposure to benchmark material could contribute, so these results do not establish that training-data contamination caused the gap.
Finally, the authors deliberately assessed direct problem solving rather than allowing tools, so the findings do not establish how well a model combined with external tools would solve the puzzles. Together with the narrow puzzle domain and limited access to internal model states, this bounds the paper’s broader conclusions about reasoning.
08What is the analytical or experimental setup, and is it appropriate and sufficient?
The setup is well suited to testing unaided, step-by-step puzzle solving, but it is not sufficient to establish that reasoning models lack general reasoning ability or face a fundamental limit across all tasks.
The authors tested four puzzles: transferring a stack of disks, swapping colored checkers, crossing a river under safety constraints, and rearranging blocks. They made each puzzle harder by increasing its size, allowing them to track performance within the same kind of problem.
They compared reasoning models, which produce extended intermediate thinking text before answering, with standard model counterparts, and also tested other reasoning models on final answers. They used 25 samples per model for each puzzle instance and complexity level, filtering for responses that followed the requested format.
The models had to provide complete lists of moves, which puzzle simulators checked against the rules and the goal state. Accuracy measures the share of evaluated responses that solve the puzzle correctly: a higher value means more complete solutions passed, while one incorrect move makes a solution fail.
The authors also compared models using pass@k, the chance that at least one of k attempts succeeds, under comparable budgets of tokens, which are pieces of generated text. This makes the comparison sensitive to the amount of generated text used to obtain a solution, although token count is a proxy for inference effort rather than a direct measurement of every computational resource.
Across the tested puzzles, standard models often matched or exceeded reasoning models on easy problems, reasoning models had an advantage on moderately difficult ones, and both reached near-zero accuracy on sufficiently difficult ones. These results support a difficulty-dependent advantage for extended thinking in these tests, not a guarantee that either approach will solve a harder instance.
The authors measured reasoning effort by the number of thinking tokens and found that it often rose with puzzle difficulty before falling near the point of accuracy collapse, despite unused generation capacity. They interpret this pattern as a scaling limit, but it does not by itself establish a fundamental limit on every possible way to allocate more effort.
To examine the models’ written thinking, the authors extracted unique attempted move lists from Claude’s reasoning traces and checked them with the simulators. On easier problems, correct attempts often appeared early; on moderately harder ones, they tended to appear later; and on the hardest problems, the analysis found no correct attempts.
In my assessment, that trace analysis is informative but incomplete: the prompts requested full move lists even during thinking, and the extraction retained only recognizable, unique lists. The paper also notes that its mainly black-box access prevents a mechanistic analysis of the models’ internal states.
Providing a solving algorithm did not reliably prevent collapse on the disk and checker puzzles, which the authors take as evidence of difficulty executing procedures as well as finding them. In my assessment, the checker test is qualified by its supplied pseudocode: the main procedure ignores results returned by attempted recursive moves, although an accompanying note tells the model to use backtracking.
The authors checked two alternative explanations for failure: first errors often occurred well before the end of a required solution, and choosing the most likely next token instead of sampling did not eliminate collapse. These checks make a simple generation-limit or random-sampling explanation less convincing, though the authors acknowledge that longer sequences can reduce the chance of a completely correct answer.
In my assessment, filtering for the requested format is another qualification: it can exclude failures from the reported results, and the paper does not report enough about excluded samples to determine the effect. More broadly, the authors acknowledge that these structured puzzles cover only a narrow slice of reasoning and that their precise simulator checks may not transfer to open-ended problems.
The study deliberately assesses models without tools, so its results also do not measure how well the models would solve these puzzles using external code. Overall, the evidence is strong for limitations in the tested models’ unaided puzzle performance, but broader claims about reasoning in general remain unestablished.
09Are there identifiable gaps in the logic, proof, or interpretation?
Yes: the paper documents substantial failures on its tested puzzles, but its evidence does not conclusively establish all of its broader interpretations of those failures.
The authors compare reasoning models, which produce extended thinking before an answer, with versions or related models that do not use extended thinking. They report that, at comparable budgets of generated text, non-thinking models often do better on easier puzzles, reasoning models have an advantage at medium difficulty, and both approach zero accuracy on the hardest tested instances. This establishes a useful comparison under token budgets, but matching tokens—units of generated text—does not by itself demonstrate that the models used equal amounts of hardware computation.
The tests required complete lists of valid moves, so they measure reliable execution and output as well as the ability to find a strategy. The authors report that failures often occurred well before the end of a solution and that some short puzzles also caused collapse, which argues against explaining every failure simply as running out of space for a long answer. Nevertheless, failure to produce a complete list does not by itself show that a model lacks every form of understanding or could not solve the problem in a different setting.
To investigate whether random selection of output caused collapse, the authors also tested DeepSeek-R1 using greedy generation, which selects the most likely next piece of text rather than sampling among alternatives. Collapse still occurred, but its point changed substantially in Blocks World: the authors report collapse at four blocks with greedy generation and at 30 with sampling. This supports the conclusion that random sampling is not necessary for failure, but it does not prove that errors accumulating over a sequence play no important role.
The authors measure reasoning effort by the number of thinking tokens: a larger count means more generated thinking text. They find that this count often rises and then falls as puzzles become harder, even when the models have unused generation capacity. That is evidence about how the tested models allocated their output, not proof of an inherent limit on what additional computation, different prompts, or different stopping rules could achieve.
The reported results use 25 samples per model and puzzle instance, and the authors filter responses for the requested format. The paper does not specify how many responses were excluded or how exclusions affected the accuracy denominator, leaving the accuracy of all responses a user might receive unclear. Moreover, near-zero success in these samples does not prove that success is impossible for every attempt.
The authors also examine thinking traces—the text models produce while working—and find that correct candidate solutions tend to appear early on simple puzzles and later on moderately difficult ones. Their extractor records only the first occurrence of each identical move sequence, so its timing distributions describe unique extracted candidates rather than every time a model considered or repeated a solution. The patterns are informative about the analyzed text, but the authors acknowledge that black-box access limits investigation of the models’ internal mechanisms.
The authors report that supplying solution algorithms did not meaningfully prevent collapse on Tower of Hanoi or Checker Jumping, supporting a concern about executing procedures as well as discovering them. However, the supplied Checker Jumping pseudocode does not pass a successful recursive result back through its main loop and does not itself check that a jump crosses an opposite-colored checker, although its accompanying note asks the model to enforce that rule. Consequently, that experiment shows what happened with the guidance actually supplied, but it is not a clean test of whether a complete, correct executable algorithm would help on that puzzle.
Finally, the authors suggest that differences in training familiarity may help explain why models handle longer Tower of Hanoi sequences than some shorter River Crossing sequences. They do not establish the models’ training exposure, and they themselves caution that relative difficulty across puzzles cannot be determined from computational complexity alone without knowing the training distribution. The strongest conclusion therefore remains that these models showed limited reliability on the tested planning tasks, not that the paper has settled the nature of their reasoning in general.
10What is a useful next step, and is it proposed by the authors or suggested by the reader?
A useful next step, suggested here rather than explicitly proposed by the authors, is to test whether the paper’s puzzle findings also appear in questions that require combining facts from documents.
The authors tested reasoning models—language models that produce a detailed thinking process before answering—on puzzles of varying difficulty. They report that the fraction of correct answers fell to near zero beyond a difficulty level that differed by model. Near that point, models sometimes generated fewer thinking tokens, or pieces of text in their thinking process, even when they had room to generate more. The authors use the number of thinking tokens as a measure of reasoning effort: a higher count means more generated thinking text.
For the proposed test, researchers could ask questions about supplied documents that require combining different numbers of facts, then check the answers and count thinking tokens at each level.
The paper does not establish whether its results will extend to such tasks: the authors caution that puzzles cover a narrow slice of reasoning and that precise step-by-step validation may be harder in less structured domains.
Explanations and excerpts adapted by TimeZyme from Parshin Shojaee, Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton, Samy Bengio and Mehrdad Farajtabar. The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity. CC BY 4.0.