Stop When Enough: Adaptive Early-Stopping for Chain-of-Thought Reasoning

2025/10/11 by Sun, Renliang, Cheng, Wei, Li, Dawei +2 · 4 citations
#Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2510.10103

Abstract

Chain-of-Thought (CoT) reasoning has driven recent gains of large language models (LLMs) on reasoning-intensive tasks by externalizing intermediate steps. However, excessive or redundant reasoning -- so-called overthinking -- can increase inference costs and lead LLMs toward incorrect conclusions. In this paper, we present REFRAIN (\underlineREFlective-\underlineRedundancy for \underlineAdaptive \underlineINference), a training-free framework that adaptively determines when to stop reasoning to mitigate overthinking. REFRAIN integrates a two-stage stop discriminator to identify reflective yet redundant reasoning and a sliding-window Upper Confidence Bound (SW-UCB) multi-armed bandit controller to dynamically adjust stopping thresholds according to problem difficulty without supervision or fine-tuning. Across four representative benchmarks and two model families, REFRAIN reduces token usage by 20-55% while maintaining or improving accuracy compared to standard CoT prompting. Extensive ablation and robustness analyses demonstrate its stability across models, scorers, and prompt variations. In summary, our findings highlight when-to-stop as a new and practical axis of test-time scaling -- enabling models to reason not just more, but just enough.

Citations

• Efficient Reasoning for Large Reasoning Language Models via Certainty-Guided Reflection Suppression
• Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
• Inverse Scaling in Test-Time Compute
• SmartThinker: Learning to Compress and Preserve Reasoning by Step-Level Length Control
• AALC: Large Language Model Efficient Reasoning via Adaptive Accuracy-Length Control
• AlphaOne: Reasoning Models Thinking Slow and Fast at Test Time
• DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning Models
• SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoning
• ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoning
• Dynamic Early Exit in Reasoning Models
• Reasoning Models Can Be Effective Without Thinking
• Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Time
• Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?
• CoT-Valve: Length-Compressible Chain-of-Thought Tuning
• The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Tasks
• Demystifying Long Chain-of-Thought Reasoning in LLMs
• s1: Simple test-time scaling
• The Lessons of Developing Process Reward Models in Mathematical Reasoning
• AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modeling
• CausalEval: Towards Better Causal Reasoning in Language Models
• Can LLMs Learn from Previous Mistakes? Investigating LLMs' Errors to Boost for Reasoning
• Large Language Models Struggle with Unreasonability in Math Problems
• Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning
• GPQA: A Graduate-Level Google-Proof Q&A Benchmark
• Measuring Mathematical Problem Solving With the MATH Dataset
• HuggingFace's Transformers: State-of-the-art Natural Language Processing
• On Upper-Confidence Bound Policies for Non-Stationary Bandit Problems
• CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge

Cited by

• Asynchronous Reasoning: Training-Free Interactive Thinking LLMs
• Mull-Tokens: Modality-Agnostic Latent Thinking
• When to Think and When to Look: Uncertainty-Guided Lookback
• Online Reasoning Calibration: Test-Time Training Enables Generalizable Conformal LLM Reasoning

Related

• Stop Spinning Wheels: Mitigating LLM Overthinking via Mining Patterns for Early Reasoning Exit
• Early Stopping Chain-of-thoughts in Large Language Models
• Dynamic Early Exit in Reasoning Models
• Reconsidering Overthinking: Penalizing Internal and External Redundancy in CoT Reasoning
• ThinkLess: A Training-Free Inference-Efficient Method for Reducing Reasoning Redundancy