Can a Small Model Learn to Look Before It Leaps? Dynamic Learning and Proactive Correction for Hallucination Detection

2025/11/08 by Bao, Zepeng, Zhou, Shen, Pi, Qiankun +5
Computer Science · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning and Algorithms #Security and Verification in Computing

paper · doi:10.48550/arxiv.2511.05854

openalex publication_date 2025/11/08 · openalex created_date 2025/11/12 · openalex updated_date 2026/07/28

Abstract

Hallucination in large language models (LLMs) remains a critical barrier to their safe deployment. For hallucination detection to be practical in real-world scenarios, the use of efficient small models is essential to ensure low latency and minimal resource consumption. However, existing methods rely on fixed verification strategies, where simply tuning small models to mimic fixed verification trajectories fails to capture the adaptability required for diverse hallucination patterns, thereby inducing planning instability. To address this limitation, we propose a ``Learning to Evaluate and Adaptively Plan'' (LEAP) framework, which shifts hallucination detection from fixed execution to dynamic strategy learning. Specifically, LEAP first employs a powerful teacher model to iteratively explore and refine verification strategies through a failure-driven loop. This dynamic planning capability is then distilled into an efficient student model, augmented by a novel proactive correction mechanism that enables the model to evaluate and optimize its verification strategy before execution. Experiments on three benchmarks demonstrate that LEAP outperforms state-of-the-art methods, offering an effective and scalable solution for reliable hallucination detection.

Citations

• FIRE: Fact-checking with Iterative Retrieval and Verification
• HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detection
• The Llama 3 Herd of Models
• Small Agent Can Also Rock! Empowering Small Language Models as Hallucination Detector
• MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
• INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection
• Hallucination Detection and Hallucination Mitigation: An Investigation
• LLMLight: Large Language Models as Traffic Signal Control Agents
• AgentTuning: Enabling Generalized Agent Abilities for LLMs
• LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examples
• Efficient Memory Management for Large Language Model Serving with PagedAttention
• FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios
• A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation
• FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
• Reflexion: Language Agents with Verbal Reinforcement Learning
• Distributing Accountability, Not Capability: Phase Separation and the LLM Workflow Quadrant in Autonomous AI Agent Architectures
• Out-of-Distribution Detection and Selective Generation for Conditional Language Models
• Billion-scale similarity search with GPUs

Related

• Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Models
• Teaming LLMs to Detect and Mitigate Hallucinations
• Theoretical Foundations and Mitigation of Hallucination in Large Language Models
• Small Agent Can Also Rock! Empowering Small Language Models as Hallucination Detector
• Detecting Hallucinations in Large Language Model Generation: A Token Probability Approach