Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!

Authors: Subbarao Kambhampati et al. — the ASU reasoning group (acknowledgments cite DARPA/ONR/DoD grants and ASU visits; self-citations span Valmeekam, Palod, Bhambri, Samineni, Stechly, Kalwar, Bhat). arXiv: 2504.09762v4, April 2025.

A position paper arguing that the field's habit of calling intermediate tokens "reasoning traces" or "thinking traces" is a dangerous anthropomorphization, not a harmless metaphor. The authors define intermediate tokens narrowly as the unfiltered output an LLM emits before its final answer — distinct from post-facto rationalizations like the sanitized summaries OpenAI shows for o1.

The core claim is that trace semantics are largely decoupled from answer correctness. Across the group's experiments — transformers trained on A search traces for maze-solving, and Llama/Qwen models fine-tuned on QA — trace validity correlates only loosely with solution accuracy, breaks down out-of-distribution, and can be wrong* (swapped, noisy, semantically incorrect) while the model still improves. RL post-training (GRPO) improves answer accuracy without improving trace validity, because it rewards only the answer.

Two further corollaries are attacked. Trace length is not "thinking effort": RL structurally incentivizes longer traces (uniform credit assignment), so DeepSeek R1's "learning to reason = longer traces" was a misattribution. And showing traces increases false trust: a human-subject study found users more likely to accept incorrect answers when shown traces or their summaries.

The alternative view: intermediate tokens are best understood as learned prompt augmentations — a scaffold the model uses to steer its context toward its training distribution. They need not be human-interpretable at all. The call to action: stop reading semantics into traces; verify solutions directly with third-party or problem-specific verifiers, especially in safety-critical settings.