Explainable Chain-of-Thought Reasoning: An Empirical Analysis on State-Aware Reasoning Dynamics

2025/08/29 by Yu, Sheldon, Xiong, Yuxin, Wu, Junda +6 · 1 citation
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2509.00190

Abstract

Recent advances in chain-of-thought (CoT) prompting have enabled large language models (LLMs) to perform multi-step reasoning. However, the explainability of such reasoning remains limited, with prior work primarily focusing on local token-level attribution, such that the high-level semantic roles of reasoning steps and their transitions remain underexplored. In this paper, we introduce a state-aware transition framework that abstracts CoT trajectories into structured latent dynamics. Specifically, to capture the evolving semantics of CoT reasoning, each reasoning step is represented via spectral analysis of token-level embeddings and clustered into semantically coherent latent states. To characterize the global structure of reasoning, we model their progression as a Markov chain, yielding a structured and interpretable view of the reasoning process. This abstraction supports a range of analyses, including semantic role identification, temporal pattern visualization, and consistency evaluation.

Citations

• CTRLS: Chain-of-Thought Reasoning via Latent State-Transition
• Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering
• Beyond Words: A Latent Memory Approach to Internal Reasoning in LLMs
• Rethinking External Slow-Thinking: From Snowball Errors to Probability of Correct Reasoning
• Reasoning Beyond Words ? Exploring framework for hidden state reasoning
• Gemma 2: Improving Open Language Models at a Practical Size
• Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps
• SpinQuant: LLM quantization with learned rotations
• Towards Uncovering How Large Language Model Works: An Explainability Perspective
• Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models
• Does Physical Adversarial Example Really Matter to Autonomous Driving? Towards System-Level Effect of Adversarial Object Evasion Attack
• Analyzing Chain-of-Thought Prompting in Large Language Models via Gradient-based Feature Attributions
• Towards Understanding In-Context Learning with Contrastive Demonstrations and Saliency Maps
• Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting
• Large Language Models are Zero-Shot Reasoners
• Training Verifiers to Solve Math Word Problems
• Measuring Mathematical Problem Solving With the MATH Dataset
• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
• CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge

Cited by

• CTRLS: Chain-of-Thought Reasoning via Latent State-Transition
• How Hard Does It Think? Analyzing Step-Aware Reasoning Energy in LLM Chain-of-Thought Trajectories

Related

• A Statistical Physics of Language Model Reasoning
• Modeling Hierarchical Thinking in Large Reasoning Models
• CTRLS: Chain-of-Thought Reasoning via Latent State-Transition
• How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning
• State over Tokens: Characterizing the Role of Reasoning Tokens