Hallucination Detection: Robustly Discerning Reliable Answers in Large Language Models

2024/07/04 by Yuyan Chen, Chen, Yuyan, Qiang Fu +15 · 7 citations
Computer Science · Social Sciences · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Misinformation and Its Impacts #Seismology and Earthquake Studies #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2407.04121

openalex publication_date 2024/07/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large Language Models (LLMs) have gained widespread adoption in various natural language processing tasks, including question answering and dialogue systems. However, a major drawback of LLMs is the issue of hallucination, where they generate unfaithful or inconsistent content that deviates from the input source, leading to severe consequences. In this paper, we propose a robust discriminator named RelD to effectively detect hallucination in LLMs' generated answers. RelD is trained on the constructed RelQA, a bilingual question-answering dialogue dataset along with answers generated by LLMs and a comprehensive set of metrics. Our experimental results demonstrate that the proposed RelD successfully detects hallucination in the answers generated by diverse LLMs. Moreover, it performs well in distinguishing hallucination in LLMs' generated answers from both in-distribution and out-of-distribution datasets. Additionally, we also conduct a thorough analysis of the types of hallucinations that occur and present valuable insights. This research significantly contributes to the detection of reliable answers generated by LLMs and holds noteworthy implications for mitigating hallucination in the future work.

Cited by

• MindFuse: Towards GenAI Explainability in Marketing Strategy Co-Creation
• MoM: Mixtures of Scenario-Aware Document Memories for Retrieval-Augmented Generation Systems
• Confidence-Aware Routing for Large Language Model Reliability Enhancement: A Multi-Signal Approach to Pre-Generation Hallucination Mitigation
• Bounded PCTL Model Checking of Large Language Model Outputs
• Control the Temperature: Selective Sampling for Diverse and High-Quality LLM Outputs
• Decomposing and Revising What Language Models Generate
• Efficient Knowledge Probing of Large Language Models by Adapting Pre-trained Embeddings

Related

• Hallucination Detection with Small Language Models
• Unsupervised Real-Time Hallucination Detection based on the Internal States of Large Language Models
• The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models
• Ask a Local: Detecting Hallucinations With Specialized Model Divergence
• DelucionQA: Detecting Hallucinations in Domain-specific Question Answering