LLMs Will Always Hallucinate, and We Need to Live With This

2024/09/09 by Sourav Banerjee, Ayushi Agarwal, Banerjee, Sourav +3 · 27 voices · 34 citations
Computer Science · Social Sciences · #Law, AI, and Intellectual Property #Legal Education and Practice Innovations #Legal Systems and Judicial Processes #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2409.05746

openalex publication_date 2024/09/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

As Large Language Models become more ubiquitous across domains, it becomes important to examine their inherent limitations critically. This work argues that hallucinations in language models are not just occasional errors but an inevitable feature of these systems. We demonstrate that hallucinations stem from the fundamental mathematical and logical structure of LLMs. It is, therefore, impossible to eliminate them through architectural improvements, dataset enhancements, or fact-checking mechanisms. Our analysis draws on computational theory and Godel's First Incompleteness Theorem, which references the undecidability of problems like the Halting, Emptiness, and Acceptance Problems. We demonstrate that every stage of the LLM process-from training data compilation to fact retrieval, intent classification, and text generation-will have a non-zero probability of producing hallucinations. This work introduces the concept of Structural Hallucination as an intrinsic nature of these systems. By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated.

Cited by

• An approach to systemic risks of AI through the lens of emergence, collective action problems, and externalities
• Hallucinations Undermine Trust; Metacognition is a Way Forward
• Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agents
• The Epistemological Consequences of Large Language Models: Rethinking collective intelligence and institutional knowledge
• Neuro-Symbolic Control with Large Language Models for Language-Guided Spatial Tasks
• Plausibility as Failure: How LLMs and Humans Co-Construct Epistemic Error
• A Scientific Reasoning Model for Organic Synthesis Procedure Generation
• The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines
• A Concise Review of Hallucinations in LLMs and their Mitigation
• The Oracle and The Prism: A Decoupled and Efficient Framework for Generative Recommendation Explanation
• Flash-Fusion: Enabling Expressive, Low-Latency Queries on IoT Sensor Streams with LLMs
• Maestro: Learning to Collaborate via Conditional Listwise Policy Optimization for Multi-Agent LLMs
• Slurry-as-a-Service: A Modest Proposal on Scalable Pluralistic Alignment for Nutrient Optimization
• Mitigating Hallucination in Large Language Models (LLMs): An Application-Oriented Survey on RAG, Reasoning, and Agentic Systems
• HACK: Hallucinations Along Certainty and Knowledge Axes
• KoSimpleQA: A Korean Factuality Benchmark with an Analysis of Reasoning LLMs
• MOSAIC: Multi-agent Orchestration for Task-Intelligent Scientific Coding
• Large Language Models Hallucination: A Comprehensive Survey
• Hallucination is Inevitable for LLMs with the Open World Assumption
• Are Hallucinations Bad Estimations?
• Reward Evolution with Graph-of-Thoughts: A Bi-Level Language Model Framework for Reinforcement Learning
• DPCheatSheet: Using Worked and Erroneous LLM-usage Examples to Scaffold Differential Privacy Implementation
• Deploying AI for Signal Processing education: Selected challenges and intriguing opportunities
• Proof-Carrying Numbers (PCN): A Protocol for Trustworthy Numeric Answers from LLMs via Claim Verification
• Charting the Future of Scholarly Knowledge with AI: A Community Perspective
• Mitigating Hallucinations in Large Language Models via Causal Reasoning
• Considering the ethics of large machine learning models in the chemical sciences
• Dean of LLM Tutors: Exploring Comprehensive and Automated Evaluation of LLM-generated Educational Feedback via LLM Feedback Evaluators
• Towards Reliable Generative AI-Driven Scaffolding: Reducing Hallucinations and Enhancing Quality in Self-Regulated Learning Support
• Studying Disinformation Narratives on Social Media with LLMs and Semantic Similarity
• Mapping the susceptibility of large language models to medical misinformation across clinical notes and social media: a cross-sectional benchmarking analysis
• Aligning Knowledge Graphs and Language Models for Factual Accuracy
• AI hallucinations can’t be stopped — but these techniques can limit their damage
• Knowledge Conceptualization Impacts RAG Efficacy

Discussions

• LLMs Will Always Hallucinate, and We Need to Live with This [hn, 291 points, 261 comments]
• Good paper on this if people are interested arxiv.org/abs/2409.05746 [bsky, 25 points, 2 comments]
• LLMs Will Always Hallucinate, and We Need to Live With This [lobsters, 13 points, 22 comments]
• A common scholarly consensus is that hallucinations may increase rather than decrease as models become more advanced - and they definitely won’t go away. It is sensible to be skeptical about this tech [bsky, 4 points, 0 comments]
• LLMs Will Always Hallucinate, and We Need to Live with This [hn, 4 points, 0 comments]
• Hallucination study: arxiv.org/abs/2409.05746 Model collapse study: www.nature.com/articles/s41... [bsky, 3 points, 1 comments]
• 💯 Gibt auch zahlreiche papers dazu: arxiv.org/abs/2409.05746 [bsky, 2 points, 0 comments]
• 할루시네이션은 없앨 수 없다고 증명된 [bsky, 1 points, 0 comments]
• This paper shows that AI will always be imperfect. It does it using Gödel's incompleteness theorems, which have been proven both theoretically and in practice. Besides that there is basically giving O [bsky, 1 points, 0 comments]
• An interesting analysis using both Godel's incompleteness theorem and the halting problem can be found here (proving that it will forever been inaccurate and hallucinate). I'd say that the lower error [bsky, 1 points, 0 comments]
• Hallucinaties zijn fundamenteel probleem en blijven dat, 3 years down the line. Er is zelfs een (voorzichtig) bewijs dat LLMs dat probleem altijd zullen hebben: https://arxiv.org/abs/2409.05746 AI ch [bsky, 1 points, 0 comments]
• LLMs Will Always Hallucinate, and We Need to Live with This (arxiv.org) Main Link | Discussion [bsky, 1 points, 0 comments]
• arxiv.org/abs/2409.05746 [bsky, 1 points, 0 comments]
• Die Qualität einiger wissenschaftlicher Artikel zu KI und LLM auf den Preprint-Servern ist erschreckend: "LLMs Will Always Hallucinate, and We Need to Live With This" (arxiv.org/abs/2409.05746v1) mit [bsky, 1 points, 1 comments]
• Fwiw arxiv.org/abs/2409.05746 [bsky, 1 points, 0 comments]
• [2409.05746] LLMs Will Always Hallucinate, and We Need to Live With This [bsky, 0 points, 0 comments]
• "Need to Live"もやや意識してそうだな いや、あんま似てないか LLMs Will Always Hallucinate, and We Need to Live With This arxiv.org/abs/2409.057... 構造的幻覚(Structural Hallucinations)について データセットは本質的に不完全でありLLMからハルシネーションを軽減するのは不 [bsky, 0 points, 0 comments]
• I have yet to read this but the abstract is consistent with my intuition. Looks like a long slog but I hope to get to it soon arxiv.org/abs/2409.057... [bsky, 0 points, 0 comments]
• What I don't understand is why LLM's can't just give confidence levels like classic intent engines? Ostensibly, Google Gemini via Vertex seems to do this fairly well..? [bsky, 0 points, 0 comments]
• This is the paper I'm referencing arxiv.org/abs/2409.05746 But researchers have been saying that pretty much all signs point to the current LLM strategy not working forever for a while now. [bsky, 0 points, 1 comments]
• https://bsky.app/profile/thalgar.bsky.social/post/3mcsjugq6v22h [bsky, 0 points, 0 comments]
• LLMs Will Always Hallucinate, and We Need to Live with This [bsky, 0 points, 0 comments]
• LLMs will always hallucinate. arxiv.org/abs/2409.057... [bsky, 0 points, 0 comments]
• LLM Hallucination을 피할수 없다는 연구 arxiv.org/pdf/2409.05746 [bsky, 0 points, 1 comments]
• It's a feature not a bug - important paper. "This work argues that hallucinations in language models are not just occasional errors but an inevitable feature of these systems." arxiv.org/abs/2409.0 [bsky, 0 points, 0 comments]
• The funniest thing to me about the AI bubble is that everyone went all in on LLMs because they're easier other AI models assuming that they could eventually fix the flaws and hallucinations; and after [bsky, 0 points, 0 comments]
• arxiv.org/abs/2409.05746 If you don't understand that hallucinations are mathematically impossible to eliminate or even meaningfully mitigate, then you don't really understand how LLMs work. [bsky, 0 points, 1 comments]

Related

• Hallucination is Inevitable: An Innate Limitation of Large Language Models
• Hallucinations are inevitable but can be made statistically negligible
• Large Language Models Hallucination: A Comprehensive Survey
• A comprehensive taxonomy of hallucinations in Large Language Models
• Distinguishing Ignorance from Error in LLM Hallucinations