Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future

2023/09/27 by Zheng Chu, Jingchang Chen, Chu, Zheng +16 · 27 citations
Computer Science · #Advanced Graph Neural Networks #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #IoT and Edge/Fog Computing

paper · pdf · doi:10.48550/arxiv.2309.15402

openalex publication_date 2023/09/27 · openalex created_date 2023/09/30 · openalex updated_date 2026/07/28

Abstract

Reasoning, a fundamental cognitive process integral to human intelligence, has garnered substantial interest within artificial intelligence. Notably, recent studies have revealed that chain-of-thought prompting significantly enhances LLM's reasoning capabilities, which attracts widespread attention from both academics and industry. In this paper, we systematically investigate relevant research, summarizing advanced methods through a meticulous taxonomy that offers novel perspectives. Moreover, we delve into the current frontiers and delineate the challenges and future directions, thereby shedding light on future research. Furthermore, we engage in a discussion about open questions. We hope this paper serves as an introduction for beginners and fosters future research. Resources have been made publicly available at https://github.com/zchuz/CoT-Reasoning-Survey

Cited by

• Understanding Chain-of-Thought in Large Language Models via Topological Data Analysis
• CienaLLM: Generative Climate-Impact Extraction from News Articles with Autoregressive LLMs
• To Think or Not to Think: The Hidden Cost of Meta-Training with Excessive CoT Examples
• A Flexible Multi-Agent LLM-Human Framework for Fast Human Validated Tool Building
• From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Models
• Vocabulary In-Context Learning in Transformers: Benefits of Positional Encoding
• Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generation
• Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism
• Verifying Chain-of-Thought Reasoning via Its Computational Graph
• Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMs
• CIR-CoT: Towards Interpretable Composed Image Retrieval via End-to-End Chain-of-Thought Reasoning
• Towards Interpretable and Inference-Optimal COT Reasoning with Sparse Autoencoder-Guided Generation
• Making Mathematical Reasoning Adaptive
• Unveiling Chain of Step Reasoning for Vision-Language Models with Fine-grained Rewards
• ReGeS: Reciprocal Retrieval-Generation Synergy for Conversational Recommender Systems
• Correlation or Causation: Analyzing the Causal Structures of LLM and LRM Reasoning Process
• Improving Large Language Models Function Calling and Interpretability via Guided-Structured Templates
• USB-Rec: An Effective Framework for Improving Conversational Recommendation Capability of Large Language Model
• A Survey of Reasoning and Agentic Systems in Time Series with Large Language Models
• Learning from Diverse Reasoning Paths with Routing and Collaboration
• Less is More Tokens: Efficient Math Reasoning via Difficulty-Aware Chain-of-Thought Distillation
• Vis-CoT: A Human-in-the-Loop Framework for Interactive Visualization and Intervention in LLM Chain-of-Thought Reasoning
• MIRAGE: Scaling Test-Time Inference with Parallel Graph-Retrieval-Augmented Reasoning Chains
• Automated Optimization Modeling through Expert-Guided Large Language Model Reasoning
• Learning by Teaching: Engaging Students as Instructors of Large Language Models in Computer Science Education
• DeepVIS: Bridging Natural Language and Data Visualization Through Step-wise Reasoning
• AgriEval: A Comprehensive Chinese Agricultural Benchmark for Large Language Models

Related

• Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism
• Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
• CoAT: Chain-of-Associated-Thoughts Framework for Enhancing Large Language Models Reasoning
• Chain of Evidences and Evidence to Generate: Prompting for Context Grounded and Retrieval Augmented Reasoning
• Chain or tree? Re-evaluating complex reasoning from the perspective of a matrix of thought