Theoretical Foundations and Mitigation of Hallucination in Large Language Models

2025/07/20 by Esmail Gumaan, Gumaan, Esmail · 1 citation
Computer Science · Psychology · #Machine Learning in Healthcare #Mental Health via Writing #Mental Health Research Topics

paper · pdf · doi:10.48550/arxiv.2507.22915

Abstract

Hallucination in Large Language Models (LLMs) refers to the generation of content that is not faithful to the input or the real-world facts. This paper provides a rigorous treatment of hallucination in LLMs, including formal definitions and theoretical analyses. We distinguish between intrinsic and extrinsic hallucinations, and define a hallucination risk for models. We derive bounds on this risk using learning-theoretic frameworks (PAC-Bayes and Rademacher complexity). We then survey detection strategies for hallucinations, such as token-level uncertainty estimation, confidence calibration, and attention alignment checks. On the mitigation side, we discuss approaches including retrieval-augmented generation, hallucination-aware fine-tuning, logit calibration, and the incorporation of fact-verification modules. We propose a unified detection and mitigation workflow, illustrated with a diagram, to integrate these strategies. Finally, we outline evaluation protocols for hallucination, recommending datasets, metrics, and experimental setups to quantify and reduce hallucinations. Our work lays a theoretical foundation and practical guidelines for addressing the crucial challenge of hallucination in LLMs.

Citations

• Hallucination is Inevitable: An Innate Limitation of Large Language Models
• A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
• Chain-of-Verification Reduces Hallucination in Large Language Models
• SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
• Language Models (Mostly) Know What They Know
• Training language models to follow instructions with human feedback
• Survey of Hallucination in Natural Language Generation
• WebGPT: Browser-assisted question-answering with human feedback
• TruthfulQA: Measuring How Models Mimic Human Falsehoods
• Understanding Factuality in Abstractive Summarization with FRANK: A\n Benchmark for Factuality Metrics
• The Curious Case of Hallucinations in Neural Machine Translation
• How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering
• Selective Question Answering under Domain Shift
• Language Models are Few-Shot Learners
• On Faithfulness and Factuality in Abstractive Summarization
• Asking and Answering Questions to Evaluate the Factual Consistency of Summaries
• Calibration of Pre-trained Transformers
• REALM: Retrieval-Augmented Language Model Pre-Training
• Evaluating the Factual Consistency of Abstractive Text Summarization
• The Curious Case of Neural Text Degeneration
• FEVER: a large-scale dataset for Fact Extraction and VERification
• On Calibration of Modern Neural Networks
• Attention Is All You Need
• Six Challenges for Neural Machine Translation
• Get To The Point: Summarization with Pointer-Generator Networks
• What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
• A Neural Conversational Model
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• TRUE: Re-evaluating Factual Consistency Evaluation
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Cited by

• Explainable Knowledge Graph Retrieval-Augmented Generation (KG-RAG) with KG-SMILE

Related

• Large Language Models Hallucination: A Comprehensive Survey
• 🧜Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models
• Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
• A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
• The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models