One SPACE to Rule Them All: Jointly Mitigating Factuality and Faithfulness Hallucinations in LLMs

2025/06/05 by Pengbo Wang, Wang, Pengbo, Chaozhuo Li +9 · 1 citation
Computer Science · #68T50 #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Digital Rights Management and Security #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.2506.11088

openalex publication_date 2025/06/05 · openalex created_date 2025/10/11 · openalex updated_date 2026/08/01

Abstract

LLMs have demonstrated unprecedented capabilities in natural language processing, yet their practical deployment remains hindered by persistent factuality and faithfulness hallucinations. While existing methods address these hallucination types independently, they inadvertently induce performance trade-offs, as interventions targeting one type often exacerbate the other. Through empirical and theoretical analysis of activation space dynamics in LLMs, we reveal that these hallucination categories share overlapping subspaces within neural representations, presenting an opportunity for concurrent mitigation. To harness this insight, we propose SPACE, a unified framework that jointly enhances factuality and faithfulness by editing shared activation subspaces. SPACE establishes a geometric foundation for shared subspace existence through dual-task feature modeling, then identifies and edits these subspaces via a hybrid probe strategy combining spectral clustering and attention head saliency scoring. Experimental results across multiple benchmark datasets demonstrate the superiority of our approach.

Citations

• The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs
• Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledge
• Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models
• Cognitive Activation and Chaotic Dynamics in Large Language Models: A Quasi-Lyapunov Analysis of Reasoning Mechanisms
• DSVD: Dynamic Self-Verify Decoding for Faithful Generation in Large Language Models
• Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projection
• LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations
• On the Universal Truthfulness Hyperplane Inside LLMs
• Improving Faithfulness of Large Language Models in Summarization via Sliding Generation and Self-Consistency
• TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space
• INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection
• Self-Contrast: Better Reflection Through Inconsistent Solving Perspectives
• Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuning
• Efficient Representation of the Activation Space in Deep Neural Networks
• Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patching
• A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
• DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
• Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models
• 🧜Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models
• Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
• Trusting Your Evidence: Hallucinate Less with Context-aware Decoding
• Reflexion: Language Agents with Verbal Reinforcement Learning
• GPT-4 Technical Report
• Discovering Latent Knowledge in Language Models Without Supervision
• EViT: Privacy-Preserving Image Retrieval via Encrypted Vision Transformer in Cloud Computing
• Survey of Hallucination in Natural Language Generation
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• TruthfulQA: Measuring How Models Mimic Human Falsehoods
• AdsGNN: Behavior-Graph Augmented Relevance Modeling in Sponsored Search
• Steer LLM Latents for Hallucination Detection

Cited by

• Bridging External and Parametric Knowledge: Mitigating Hallucination of LLMs with Shared-Private Semantic Synergy in Dual-Stream Knowledge

Related

• Constructing Benchmarks and Interventions for Combating Hallucinations in LLMs
• Banishing LLM Hallucinations Requires Rethinking Generalization
• FactCheckmate: Preemptively Detecting and Mitigating Hallucinations in LMs
• LLM Internal States Reveal Hallucination Risk Faced With a Query
• HalluShift: Measuring Distribution Shifts towards Hallucination Detection in LLMs