Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
2023/09/03 by Yue Zhang, Yafu Li, Zhang, Yue +29 · 1 voice · 192 citations
Computer Science · Medicine · #Artificial Intelligence (cs.AI) #COVID-19 diagnosis using AI #Computation and Language (cs.CL) #Computers and Society (cs.CY) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning in Healthcare #Topic Modeling #cs.AI #cs.CL #cs.CY #cs.LG
paper · pdf · doi:10.48550/arxiv.2309.01219
openalex publication_date 2023/09/03 · arxiv published 2023/09/03 · arxiv updated 2025/09/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Abstract
While large language models (LLMs) have demonstrated remarkable capabilities across a range of downstream tasks, a significant concern revolves around their propensity to exhibit hallucinations: LLMs occasionally generate content that diverges from the user input, contradicts previously generated context, or misaligns with established world knowledge. This phenomenon poses a substantial challenge to the reliability of LLMs in real-world scenarios. In this paper, we survey recent efforts on the detection, explanation, and mitigation of hallucination, with an emphasis on the unique challenges posed by LLMs. We present taxonomies of the LLM hallucination phenomena and evaluation benchmarks, analyze existing approaches aiming at mitigating LLM hallucination, and discuss potential directions for future research.
Cited by
- Lie to Me: Knowledge Graphs for Robust Hallucination Self-Detection in LLMs
- Flexible Keyword-Aware Top-k Route Search
- Expert-Grounded Automatic Prompt Engineering for Extracting Lattice Constants of High-Entropy Alloys from Scientific Publications using Large Language Models
- TRE: Training-Free Hallucination Detection for Diffusion Language Models
- Multimodal LLMs for Historical Dataset Construction from Archival Image Scans: German Patents (1877-1918)
- AIAuditTrack: A Framework for AI Security system
- MiniScope: A Least Privilege Framework for Authorizing Tool Calling Agents
- CHEM: Estimating and Understanding Hallucinations in Deep Learning for Image Processing
- KBQA-R1: Reinforcing Large Language Models for Knowledge Base Question Answering
- SimpleDevQA: Benchmarking Large Language Models on Development Knowledge QA
- Toward Faithful Retrieval-Augmented Generation with Sparse Autoencoders
- Enhancing Retrieval-Augmented Generation with Entity Linking for Educational Platforms
- Latent Debate: A Surrogate Framework for Interpreting LLM Thinking
- Generative AI Practices, Literacy, and Divides: An Empirical Analysis in the Italian Context
- Detecting AI Hallucinations in Finance: An Information-Theoretic Method Cuts Hallucination Rate by 92%
- How Far Are We from Genuinely Useful Deep Research Agents?
- Auditable Context-Aware HFMD Forecasting with Structured LLM Agents
- Beyond Component Strength: Synergistic Integration and Adaptive Calibration in Multi-Agent RAG Systems
- A Multifaceted Analysis of Negative Bias in Large Language Models through the Lens of Parametric Knowledge
- Scaling Open-Weight Large Language Models for Hydropower Regulatory Information Extraction: A Systematic Analysis
- When Bias Pretends to Be Truth: How Spurious Correlations Undermine Hallucination Detection in LLMs
- A Low-Rank Method for Vision Language Model Hallucination Mitigation in Autonomous Driving
- AGRAG: Advanced Graph-based Retrieval-Augmented Generation for LLMs
- Validity Is What You Need
- AIriskEval-edu Demo: Auditing of Pedagogical Risks in Educational Explanations
- Polistemics: Evaluating LLMs as Information Mediators in Politics & Elections
- Can Knowledge-Graph-based Retrieval Augmented Generation Really Retrieve What You Need?
- EDVD-LLaMA: Explainable Deepfake Video Detection via Multimodal Large Language Model Reasoning
- Large-scale manual curation and harmonization of metadata from metagenomic and cancer genomic repositories: challenges and solutions
- SMAGDi: Socratic Multi Agent Interaction Graph Distillation for Efficient High Accuracy Reasoning
- Mitigating Hallucination in Large Language Models (LLMs): An Application-Oriented Survey on RAG, Reasoning, and Agentic Systems
- Metadata-Driven Retrieval-Augmented Generation for Financial Question Answering
- PICOs-RAG: PICO-supported Query Rewriting for Retrieval-Augmented Generation in Evidence-Based Medicine
- M-Eval: A Heterogeneity-Based Framework for Multi-evidence Validation in Medical RAG Systems
- MAD-Fact: A Multi-Agent Debate Framework for Long-Form Factuality Evaluation in LLMs
- AutoMT: A Multi-Agent LLM Framework for Automated Metamorphic Testing of Autonomous Driving Systems
- Structured and Abstractive Reasoning on Multi-modal Relational Knowledge Images
- A Graph Signal Processing Framework for Hallucination Detection in Large Language Models
- From Memorization to Generalization: Fine-Tuning Large Language Models for Biomedical Term-to-Identifier Normalization
- Disparities in Multilingual LLM-Based Healthcare Q&A
- An Efficient Rubric-based Generative Verifier for Search-Augmented LLMs
- MedTrust-RAG: Evidence Verification and Trust Alignment for Biomedical Question Answering
- ESI: Epistemic Uncertainty Quantification via Semantic-preserving Intervention for Large Language Models
- AgentCaster: Reasoning-Guided Tornado Forecasting
- Beyond Textual CoT: Interleaved Text-Image Chains with Deep Confidence Reasoning for Image Editing
- Exposing Citation Vulnerabilities in Generative Engines
- Large Language Models Hallucination: A Comprehensive Survey
- Increasing LLM response trustworthiness using voting ensembles
- SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinations
- ReSeek: A Self-Correcting Framework for Search Agents with Instructive Rewards
- Copy-Paste to Mitigate Large Language Model Hallucinations
- TraceDet: Hallucination Detection from the Decoding Trace of Diffusion Large Language Models
- HFuzzer: Testing Large Language Models for Package Hallucinations via Phrase-based Fuzzing
- An Improved Framework for Scaling Party Positions from Texts with Transformer
- Do LLM Agents Know How to Ground, Recover, and Assess? A Benchmark for Epistemic Competence in Information-Seeking Agents
- GraphSearch: An Agentic Deep Searching Workflow for Graph Retrieval-Augmented Generation
- Are Hallucinations Bad Estimations?
- DSA, AIA, and LLMs: Approaches to conceptualizing and auditing moderation in LLM-based chatbots across languages and interfaces in the electoral contexts
- Navigating the AI era: university communication strategies and perspectives on generative AI tools
- SynBench: A Benchmark for Differentially Private Text Generation
- Constrained Prompt Enhancement for Improving Zero-Shot Generalization of Vision-Language Models
- DSCC-HS: A Dynamic Self-Reinforcing Framework for Hallucination Suppression in Large Language Models
- SIRAG: Towards Stable and Interpretable RAG with A Process-Supervised Multi-Agent Framework
- HiChunk: Evaluating and Enhancing Retrieval-Augmented Generation with Hierarchical Chunking
- Agentic System with Modal Logic for Autonomous Diagnostics
- LightAgent: Production-level Open-source Agentic AI Framework
- Harnessing Uncertainty: Entropy-Modulated Policy Gradients for Long-Horizon LLM Agents
- TraceRAG: A LLM-Based Framework for Explainable Android Malware Detection and Behavior Analysis
- Aligning LLMs for the Classroom with Knowledge-Based Retrieval -- A Comparative RAG Study
- Investigating Symbolic Triggers of Hallucination in Gemma Models Across HaluEval and TruthfulQA
- Towards EnergyGPT: A Large Language Model Specialized for the Energy Sector
- LAMDAS: LLM as an Implicit Classifier for Domain-specific Data Selection
- Test-Time Scaling in Reasoning Models Is Not Effective for Knowledge-Intensive Tasks Yet
- From Noise to Narrative: Tracing the Origins of Hallucinations in Transformers
- LLMCDSR: Enhancing Cross-Domain Sequential Recommendation with Large Language Models
- VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Use
- EviNote-RAG: Enhancing RAG Models via Answer-Supportive Evidence Notes
- Topic Identification in LLM Input-Output Pairs through the Lens of Information Bottleneck
- Foundational Design Principles and Patterns for Building Robust and Adaptive GenAI-Native Systems
- Toward Generalized Autonomous Agents: A Neuro-Symbolic AI Framework for Integrating Social and Technical Support in Education
- Hallucinations in medical devices
- Feedback Indicators: The Alignment between Llama and a Teacher in Language Learning
- Considering the ethics of large machine learning models in the chemical sciences
- Prompt-Response Semantic Divergence Metrics for Faithfulness Hallucination and Misalignment Detection in Large Language Models
- Retrieval-Augmented Generation in Industry: An Interview Study on Use Cases, Requirements, Challenges, and Evaluation
- Beyond Prompt-Induced Lies: Investigating LLM Deception on Benign Prompts
- From Explainable to Explanatory Artificial Intelligence: Toward a New Paradigm for Human-Centered Explanations through Generative AI
- Beyond Automation: Socratic AI, Epistemic Agency, and the Implications of the Emergence of Orchestrated Multi-Agent Learning Architectures
- ASINT: Learning AS-to-Organization Mapping from Internet Metadata
- MCeT: Behavioral Model Correctness Evaluation using Large Language Models
- How Far Are AI Scientists from Changing the World?
- Policy Learning from Large Vision-Language Model Feedback without Reward Modeling
- Graph-R1: Towards Agentic GraphRAG Framework via End-to-end Reinforcement Learning
- MAAD: Automate Software Architecture Design through Knowledge-Driven Multi-Agent Collaboration
- Hallucination Detection and Mitigation with Diffusion in Multi-Variate Time-Series Foundation Models
- Automating Expert-Level Medical Reasoning Evaluation of Large Language Models
- LayerCake: Token-Aware Contrastive Decoding within Large Language Model Layers
- RALLY: Role-Adaptive LLM-Driven Yoked Navigation for Agentic UAV Swarms
- KScope: A Framework for Characterizing the Knowledge Status of Language Models
- Counterfactual Segmentation Reasoning: Diagnosing and Mitigating Pixel-Grounding Hallucination
- A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasures
- Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
- HalluRNN: Mitigating Hallucinations via Recurrent Cross-Layer Reasoning in Large Vision-Language Models
- Leveraging LLMs to Assess Tutor Moves in Real-Life Dialogues: A Feasibility Study
- "I Cannot Write This Because It Violates Our Content Policy": Understanding Content Moderation Policies and User Experiences in Generative AI Products
- Prime the search: Using large language models for guiding geometric task and motion planning by warm-starting tree search
- Enhancing Rating-Based Reinforcement Learning to Effectively Leverage Feedback from Large Vision-Language Models
- Exploring the Secondary Risks of Large Language Models
- Towards Understanding the Cognitive Habits of Large Reasoning Models
- Reducing Object Hallucination in Large Audio-Language Models via Audio-Aware Decoding
- Reasoning RAG via System 1 or System 2: A Survey on Reasoning Agentic Retrieval-Augmented Generation for Industry Challenges
- Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models
- The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs
- Can Theoretical Physics Research Benefit from Language Agents?
- Multiple-Choice Question Generation Using Large Language Models: Methodology and Educator Insights
- One SPACE to Rule Them All: Jointly Mitigating Factuality and Faithfulness Hallucinations in LLMs
- Attack Effect Model based Malicious Behavior Detection
- CDE-Mapper: Using Retrieval-Augmented Language Models for Linking Clinical Data Elements to Controlled Vocabularies
- Machine Mirages: Defining the Undefined
- Are Large Language Models Good Temporal Graph Learners?
- When LLMs Team Up: The Emergence of Collaborative Affective Computing
- Benford's Curse: Tracing Digit Bias to Numerical Hallucination in LLMs
- Contextual Candor: Enhancing LLM Trustworthiness Through Hierarchical Unanswerability Detection
- OntoRAG: Enhancing Question-Answering through Automated Ontology Derivation from Unstructured Knowledge Bases
- Encouraging Students' Responsible Use of GenAI in Software Engineering Education: A Causal Model and Two Institutional Applications
- Measuring Faithfulness and Abstention: An Automated Pipeline for Evaluating LLM-Generated 3-ply Case-Based Legal Arguments
- Guiding Generative Storytelling with Knowledge Graphs
- Reasoning Models Hallucinate More: Factuality-Aware Reinforcement Learning for Large Reasoning Models
- LoVeC: Reinforcement Learning for Better Verbalized Confidence in Long-Form Generations
- Distinguishing Fact from Fiction: Student Traits, Attitudes, and AI Hallucination Detection in Business School Assessment
- AVCD: Mitigating Hallucinations in Audio-Visual Large Language Models through Contrastive Decoding
- Iterative Corpus Refinement for Materials Property Prediction Based on Scientific Texts
- Improving Recommendation Fairness without Sensitive Attributes Using Multi-Persona LLMs
- Meta-aware Learning in text-to-SQL Large Language Model
- Do You Keep an Eye on What I Ask? Mitigating Multimodal Hallucination via Attention-Guided Ensemble Decoding
- SLearnLLM: A Self-Learning Framework for Efficient Domain-Specific Adaptation of Large Language Models
- Retrieval Augmented Generation-based Large Language Models for Bridging Transportation Cybersecurity Legal Knowledge Gaps
- GPT Editors, Not Authors: The Stylistic Footprint of LLMs in Academic Preprints
- Seeing Far and Clearly: Mitigating Hallucinations in MLLMs with Attention Causal Decoding
- O2-Searcher: A Searching-based Agent Model for Open-Domain Open-Ended Question Answering
- X-MAS: Towards Building Multi-Agent Systems with Heterogeneous LLMs
- When Do LLMs Admit Their Mistakes? Understanding The Role Of Model Belief In Retraction
- MASLab: A Unified and Comprehensive Codebase for LLM-based Multi-Agent Systems
- Teaching Audio-Aware Large Language Models What Does Not Hear: Mitigating Hallucinations through Synthesized Negative Samples
- Concept Incongruence: An Exploration of Time and Death in Role Playing
- MultiHal: Multilingual Dataset for Knowledge-Graph Grounded Evaluation of LLM Hallucinations
- Reasoning Models Better Express Their Confidence
- Divide by Question, Conquer by Agent: SPLIT-RAG with Question-Driven Graph Partitioning
- Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models
- Fine-grained Contrastive Learning for ECG-Report Alignment with Waveform Enhancement
- Reasoning Large Language Model Errors Arise from Hallucinating Critical Problem Features
- scELMo: Embeddings from Language Models are Good Learners for Single-cell Data Analysis
- Finetune-RAG: Fine-Tuning Language Models to Resist Hallucination in Retrieval-Augmented Generation
- IterKey: Iterative Keyword Generation with LLMs for Enhanced Retrieval Augmented Generation
- Optimizing Retrieval-Augmented Generation: Analysis of Hyperparameter Impact on Performance and Efficiency
- ReinboT: Amplifying Robot Visual-Language Manipulation with Reinforcement Learning
- Benchmarking Retrieval-Augmented Generation for Chemistry
- DriveSOTIF: Advancing Perception SOTIF Through Multimodal Large Language Models
- ”My AI is Lying to Me”: User-reported LLM hallucinations in AI mobile apps reviews
- Quantifying Hallucinations in Language Language Models on Medical Textbooks
- Harnessing Structured Knowledge: A Concept Map-Based Approach for High-Quality Multiple Choice Question Generation with Effective Distractors
- GhostCite: A Large-Scale Analysis of Citation Validity in the Age of Large Language Models
- IslamicLegalBench: Evaluating LLMs Knowledge and Reasoning of Islamic Law Across 1,200 Years of Islamic Pluralist Legal Traditions
- Reading Between the Tokens: Improving Preference Predictions through Mechanistic Forecasting
- MADP: A Multi-Agent Pipeline for Sustainable Document Processing with Human-in-the-Loop
- Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifiers
- UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularities
- ReasonIR: Training Retrievers for Reasoning Tasks
- Can LLMs Be Trusted for Evaluating RAG Systems? A Survey of Methods and Datasets
- Meta-Cognitive Memory Policy Optimization for Long-Horizon LLM Agents
- Towards Recursive Self-Evolving Agentic Literature Retrieval
- Modular Machine Learning: An Indispensable Path towards New-Generation Large Language Models
- UHP Detection: LVLMs have their Unique Hallucination Pattern in the Consistency Space
- HalluLens: LLM Hallucination Benchmark
- (Im)possibility of Automated Hallucination Detection in Large Language Models
- Empowering AI to Generate Better AI Code: Guided Generation of Deep Learning Projects with LLMs
- CRAVE: A Conflicting Reasoning Approach for Explainable Claim Verification Using LLMs
- Meta-Thinking in LLMs via Multi-Agent Reinforcement Learning: A Survey
- Beyond Misinformation: A Conceptual Framework for Studying AI Hallucinations in (Science) Communication
- Low-hallucination Synthetic Captions for Large-Scale Vision-Language Model Pre-training
- Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associations
- Retrieval-Augmented Generation with Conflicting Evidence
- A Scoping Review of Natural Language Processing in Addressing Medically Inaccurate Information: Errors, Misinformation, and Hallucination
- The Digital Cybersecurity Expert: How Far Have We Come?
- Can LLM feedback enhance review quality? A randomized study of 20K reviews at ICLR 2025
- HalluShift: Measuring Distribution Shifts towards Hallucination Detection in LLMs
- The Other Side of the Coin: Exploring Fairness in Retrieval-Augmented Generation
- Robust Hallucination Detection in LLMs via Adaptive Token Selection
- DeepGreen: Effective LLM-Driven Green-washing Monitoring System Designed for Empirical Testing -- Evidence from China
- Endowing Embodied Agents with Spatial Reasoning Capabilities for Vision-and-Language Navigation
- Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Survey
- Don't Let It Hallucinate: Premise Verification via Retrieval-Augmented Logical Reasoning
- Retrieval Augmented Generation with Collaborative Filtering for Personalized Text Generation
Discussions
Related