Internal Chain-of-Thought: Empirical Evidence for Layer-wise Subtask Scheduling in LLMs

2025/05/20 by Zhipeng Yang, Yang, Zhipeng, Junzhuo Li +5 · 1 voice · 5 citations
Computer Science · #Big Data and Digital Economy #Generative Adversarial Networks and Image Synthesis #Topic Modeling #cs.CL #cs.LG

paper · pdf · doi:10.48550/arxiv.2505.14530

arxiv published 2025/05/20 · arxiv updated 2025/09/28

Abstract

We show that large language models (LLMs) exhibit an internal chain-of-thought: they sequentially decompose and execute composite tasks layer-by-layer. Two claims ground our study: (i) distinct subtasks are learned at different network depths, and (ii) these subtasks are executed sequentially across layers. On a benchmark of 15 two-step composite tasks, we employ layer-from context-masking and propose a novel cross-task patching method, confirming (i). To examine claim (ii), we apply LogitLens to decode hidden states, revealing a consistent layerwise execution pattern. We further replicate our analysis on the real-world TRACE benchmark, observing the same stepwise dynamics. Together, our results enhance LLMs transparency by showing their capacity to internally plan and execute subtasks (or instructions), opening avenues for fine-grained, instruction-level activation steering.

Citations

• Implicit Reasoning in Large Language Models: A Comprehensive Survey
• Back Attention: Understanding and Enhancing Multi-Hop Reasoning in Large Language Models
• Learning Task Representations from In-Context Learning
• Task Vectors in In-Context Learning: Emergence, Formation, and Benefit
• Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?
• Do I Know This Entity? Knowledge Awareness and Hallucinations in Language Models
• Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformers
• IOPO: Empowering LLMs with Complex Instruction Following via Input-Output Preference Optimization
• Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation
• Unlocking the Future: Exploring Look-Ahead Planning Mechanistic Interpretability in Large Language Models
• Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries
• LLM-based Robot Task Planning with Exceptional Handling for General Purpose Service Robots
• Implicit In-context Learning
• Do Large Language Models Latently Perform Multi-Hop Reasoning?
• Understanding and Patching Compositional Reasoning in LLMs
• Successor Heads: Recurring, Interpretable Attention Heads In The Wild
• In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering
• In-Context Learning Creates Task Vectors
• Function Vectors in Large Language Models
• Evaluating Large Language Models at Evaluating Instruction Following
• Linearity of Relation Decoding in Transformer Language Models
• Overthinking the Truth: Understanding how Language Models Process False Demonstrations
• Language Models Implement Simple Word2Vec-style Vector Arithmetic
• Dissecting Recall of Factual Associations in Auto-Regressive Language Models
• Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small
• Measuring and Narrowing the Compositionality Gap in Language Models
• Automatic Chain of Thought Prompting in Large Language Models
• In-context Learning and Induction Heads
• Locating and Editing Factual Associations in GPT
• Language Models are Few-Shot Learners
• Word Translation Without Parallel Data

Cited by

• How Do Language Models Compose Functions?
• Implicit Reasoning in Large Language Models: A Comprehensive Survey
• Punctuation and Predicates in Language Models
• Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
• Word Recovery in Large Language Models Enables Character-Level Tokenization Robustness

Discussions

• Deze techniek verergert m.i. de mate van hallucinatie en het issue met de 'blackbox' (zie ook einde h3.3 van HRM paper) Technisch klinkt het logischer, interne (latent) processing in plaats van token- [bsky, 0 points, 1 comments]

Related

• Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition
• Layer by Layer: Uncovering Where Multi-Task Learning Happens in Instruction-Tuned Large Language Models
• DecodeShare: Tracing the Shared Subspace of LLM Decode-Time Decisions
• Task Vectors, Learned Not Extracted: Performance Gains and Mechanistic Insight
• Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledge