A Stitch in Time Saves Nine: Proactive Self-Refinement for Language Models

2025/08/18 by X. Wang, Han, Jinyi, Haiquan Zhao +18 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Handwritten Text Recognition Techniques #Natural Language Processing Techniques

paper · pdf · doi:10.48550/arxiv.2508.12903

openalex publication_date 2025/08/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recent advances in self-refinement have demonstrated significant potential for improving the outputs of large language models (LLMs) through iterative refinement. However, most existing self-refinement methods rely on a reactive process with a fixed number of iterations, making it difficult to determine the optimal timing and content of refinement based on the evolving generation context. Inspired by the way humans dynamically refine their thoughts during execution, we propose ProActive Self-Refinement (PASR), a novel method that enables LLMs to refine their outputs during the generation process. Unlike methods that regenerate entire responses, PASR proactively decides whether, when, and how to refine based on the model's internal state and evolving context. We conduct extensive experiments on a diverse set of 10 tasks to evaluate the effectiveness of PASR. Experimental results show that PASR significantly enhances problem-solving performance. In particular, on Qwen3-8B, PASR reduces average token consumption by 41.6% compared to standard generation, while also achieving an 8.2% improvement in accuracy. Our code and baselines used in the paper are available on GitHub.

Citations

• DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models
• AlphaMaze: Enhancing Large Language Models' Spatial Intelligence via GRPO
• Improve LLM-as-a-Judge Ability as a General Ability
• Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMs
• Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate
• Rethinking External Slow-Thinking: From Snowball Errors to Probability of Correct Reasoning
• Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs
• From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge
• A Survey on LLM-as-a-Judge
• Think Thrice Before You Act: Progressive Thought Refinement in Large Language Models
• Training Language Models to Self-Correct via Reinforcement Learning
• Recursive Introspection: Teaching Language Model Agents How to Self-Improve
• Can LLMs Learn from Previous Mistakes? Investigating LLMs' Errors to Boost for Reasoning
• The pitfalls of next-token prediction
• Unfamiliar Finetuning Examples Control How Language Models Hallucinate
• GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinements
• BioXP-0.5B: Explainable Medical-AI via RL-GRPO
• Small Language Model Can Self-correct
• Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations
• Learning From Mistakes Makes LLM Better Reasoner
• Large Language Models Cannot Self-Correct Reasoning Yet
• Is Self-Repair a Silver Bullet for Code Generation?
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
• Teaching Large Language Models to Self-Debug
• Self-Refine: Iterative Refinement with Self-Feedback
• Language Models can Solve Computer Tasks
• Large Language Models are Better Reasoners with Self-Verification
• LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models
• Generating Sequences by Learning to Self-Correct
• Training Verifiers to Solve Math Word Problems
• Measuring Mathematical Problem Solving With the MATH Dataset
• DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
• CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge

Cited by

• Brain-Inspired Graph Multi-Agent Systems for LLM Reasoning
• ARBITER: Reasoning Trajectory Basins and Majority Vote Failures in Test-Time Sampling

Related

• Self-Refine: Iterative Refinement with Self-Feedback
• SELF: Self-Evolution with Language Feedback
• RefineBench: Evaluating Refinement Capability of Language Models via Checklists
• Evolving LLMs' Self-Refinement Capability via Synergistic Training-Inference Optimization
• Can Large Language Models Invent Algorithms to Improve Themselves?: Algorithm Discovery for Recursive Self-Improvement through Reinforcement Learning