Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought

2025/01/08 by Violet Xiang, Xiang, Violet, Charlie Snell +25 · 17 voices · 17 citations
#cs.AI #cs.CL

paper · pdf · doi:10.48550/arxiv.2501.04682

Abstract

We propose a novel framework, Meta Chain-of-Thought (Meta-CoT), which extends traditional Chain-of-Thought (CoT) by explicitly modeling the underlying reasoning required to arrive at a particular CoT. We present empirical evidence from state-of-the-art models exhibiting behaviors consistent with in-context search, and explore methods for producing Meta-CoT via process supervision, synthetic data generation, and search algorithms. Finally, we outline a concrete pipeline for training a model to produce Meta-CoTs, incorporating instruction tuning with linearized search traces and reinforcement learning post-training. Finally, we discuss open research questions, including scaling laws, verifier roles, and the potential for discovering novel reasoning algorithms. This work provides a theoretical and practical roadmap to enable Meta-CoT in LLMs, paving the way for more powerful and human-like reasoning in artificial intelligence.

Cited by

• Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRs
• LIMO: Less is More for Reasoning
• s1: Simple test-time scaling
• Automated stereotactic radiosurgery planning using a human-in-the-loop reasoning large language model agent
• VidEmo: Affective-Tree Reasoning for Emotion-Centric Video Foundation Models
• Refining Hybrid Genetic Search for CVRP via Reinforcement Learning-Finetuned LLM
• h1: Bootstrapping LLMs to Reason over Longer Horizons via Reinforcement Learning
• Searching Meta Reasoning Skeleton to Guide LLM Reasoning
• LatentEvolve: Self-Evolving Test-Time Scaling in Latent Space
• Variational Reasoning for Language Models
• Thinking Augmented Pre-training
• Learning When to Plan: Efficiently Allocating Test-Time Compute for LLM Agents
• When Thinking Backfires: Mechanistic Insights Into Reasoning-Induced Misalignment
• SWIRL: A Staged Workflow for Interleaved Reinforcement Learning in Mobile GUI Control
• Improving Pre-Trained Vision-Language-Action Policies with Model-Based Search
• Reasoning Systems as Structured Processes: Foundations, Failures, and Formal Criteria
• Understanding Human Limits in Pattern Recognition: A Computational Model of Sequential Reasoning in Rock, Paper, Scissors

Discussions

• Learning how to think with Meta Chain-of-Thought [hn, 229 points, 75 comments]
• 🧵 1/7 New paper alert: "Meta Chain-of-Thought" introduces a framework that takes LLM reasoning beyond simple step-by-step thinking. It's like upgrading from a linear path to a full navigation system [bsky, 5 points, 6 comments]
• Towards System 2 Reasoning in LLMs: Learning How to Think Meta Chain-of-Thought [hn, 1 points, 0 comments]
• Meta Chain-of-Thought explicitly models the underlying reasoning process required to solve complex problems, incorporating elements like search, verification, and iterative refinement 💡 It aims to ac [bsky, 1 points, 0 comments]
• Learning How to Think with Meta Chain-of-Thought #HackerNews arxiv.org/abs/... [bsky, 0 points, 0 comments]
• Learning How to Think with Meta Chain-of-Thought https://arxiv.org/abs/2501.04682 https://news.ycombinator.com/item?id=42655098 [bsky, 0 points, 0 comments]
• Learning How to Think with Meta Chain-of-Thought https://arxiv.org/abs/2501.04682 [comments] [23 points] [bsky, 0 points, 0 comments]
• "This work provides a theoretical and practical roadmap to enable Meta-CoT in LLMs, paving the way for more powerful and human-like reasoning in artificial intelligence." Towards System 2 Reasoning i [bsky, 0 points, 0 comments]
• Learning How to Think with Meta Chain-of-Thought https://arxiv.org/abs/2501.04682 (https://news.ycombinator.com/item?id=42655098) [bsky, 0 points, 0 comments]
• Interesting read 🤔 Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought arxiv.org/abs/2501.04682 #MachineLearning [bsky, 0 points, 0 comments]
• arxiv.org/abs/2501.04682 [bsky, 0 points, 0 comments]
• https://bsky.app/profile/news.ycombinator.com.web.brid.gy/post/3lffiyccj6yg2 [bsky, 0 points, 0 comments]
• Learning How to Think with Meta Chain-of-Thought (arxiv.org) Main Link | Discussion [bsky, 0 points, 0 comments]
• This seems like an important paper and seems to have a good summary of previous work and rigorous empirical testing. Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thoug [bsky, 0 points, 0 comments]
• "The study also raises intriguing questions for future exploration, such as the role of verifiers (mechanisms to check reasoning accuracy), how scaling impacts reasoning abilities, and whether AI migh [bsky, 0 points, 0 comments]
• Learning How to Think with Meta Chain-of-Thought https://arxiv.org/abs/2501.04682 (https://news.ycombinator.com/item?id=42655098) [bsky, 0 points, 0 comments]
• Learning How to Think with Meta Chain-of-Thought [bsky, 0 points, 0 comments]

Related

• Rethinking Chain-of-Thought from the Perspective of Self-Training
• How Likely Do LLMs with CoT Mimic Human Reasoning?
• Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
• Strategic Chain-of-Thought: Guiding Accurate Reasoning in LLMs through Strategy Elicitation
• Breaking Thought Patterns: A Multi-Dimensional Reasoning Framework for LLMs