Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models

2025/03/31 by Rui Wang, Wang, Rui, Hongru Wang +17 · 20 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Text Readability and Simplification #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2503.24377

openalex publication_date 2025/03/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recent advancements in Large Language Models (LLMs) have significantly enhanced their ability to perform complex reasoning tasks, transitioning from fast and intuitive thinking (System 1) to slow and deep reasoning (System 2). While System 2 reasoning improves task accuracy, it often incurs substantial computational costs due to its slow thinking nature and inefficient or unnecessary reasoning behaviors. In contrast, System 1 reasoning is computationally efficient but leads to suboptimal performance. Consequently, it is critical to balance the trade-off between performance (benefits) and computational costs (budgets), giving rise to the concept of reasoning economy. In this survey, we provide a comprehensive analysis of reasoning economy in both the post-training and test-time inference stages of LLMs, encompassing i) the cause of reasoning inefficiency, ii) behavior analysis of different reasoning patterns, and iii) potential solutions to achieve reasoning economy. By offering actionable insights and highlighting open challenges, we aim to shed light on strategies for improving the reasoning economy of LLMs, thereby serving as a valuable resource for advancing research in this evolving area. We also provide a public repository to continually track developments in this fast-evolving field.

Cited by

• The Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systems
• ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration
• From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Models
• One Token Embedding Is Enough to Deadlock Your Large Reasoning Model
• Implicit Reasoning in Large Language Models: A Comprehensive Survey
• CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward
• CoRE: Enhancing Metacognition with Label-free Self-evaluation in LRMs
• ReliableMath: Benchmark of Reliable Mathematical Reasoning on Large Language Models
• Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs
• Exploring and Exploiting the Inherent Efficiency within Large Reasoning Models for Self-Guided Efficiency Enhancement
• Efficient Reasoning Through Suppression of Self-Affirmation Reflections in Large Reasoning Models
• Reasoning Can Hurt the Inductive Abilities of Large Language Models
• Don't Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models
• AdaCtrl: Towards Adaptive and Controllable Reasoning via Difficulty-Aware Budgeting
• Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality
• Thinking Fast and Right: Balancing Accuracy and Reasoning Length with Adaptive Rewards
• R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Search
• Learn to Reason Efficiently with Adaptive Length-based Reward Shaping
• Let LRMs Break Free from Overthinking via Self-Braking Tuning
• Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Training

Related

• Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
• Bridging the Reasoning Gap: Small LLMs Can Plan with Generalised Strategies
• A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond
• Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models
• When Does Reasoning Matter? A Controlled Study of Reasoning's Contribution to Model Performance