Explainable Knowledge Graph Retrieval-Augmented Generation (KG-RAG) with KG-SMILE
2025/09/03 by Moghaddam, Zahra Zehtabi Sabeti, Dehghani, Zeinab, Rani, Maneeha +4
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences
paper · doi:10.48550/arxiv.2509.03626
Abstract
Generative AI, such as Large Language Models (LLMs), has achieved impressive progress but still produces hallucinations and unverifiable claims, limiting reliability in sensitive domains. Retrieval-Augmented Generation (RAG) improves accuracy by grounding outputs in external knowledge, especially in domains like healthcare, where precision is vital. However, RAG remains opaque and essentially a black box, heavily dependent on data quality. We developed a method-agnostic, perturbation-based framework that provides token and component-level interoperability for Graph RAG using SMILE and named it as Knowledge-Graph (KG)-SMILE. By applying controlled perturbations, computing similarities, and training weighted linear surrogates, KG-SMILE identifies the graph entities and relations most influential to generated outputs, thereby making RAG more transparent. We evaluate KG-SMILE using comprehensive attribution metrics, including fidelity, faithfulness, consistency, stability, and accuracy. Our findings show that KG-SMILE produces stable, human-aligned explanations, demonstrating its capacity to balance model effectiveness with interpretability and thereby fostering greater transparency and trust in machine learning technologies.
Citations
- Mitigating Hallucinations in Large Language Models via Causal Reasoning
- From Nodes to Narratives: Explaining Graph Neural Networks with LLMs and Graph Context
- Persona Vectors: Monitoring and Controlling Character Traits in Language Models
- Theoretical Foundations and Mitigation of Hallucination in Large Language Models
- KGRAG-Ex: Explainable Retrieval-Augmented Generation with Knowledge Graph-based Perturbations
- KG-TRACES: Enhancing Large Language Models with Knowledge Graph-constrained Trajectory Reasoning and Attribution Supervision
- Explaining Large Language Models with gSMILE
- G-Refer: Graph Retrieval-Augmented Large Language Model for Explainable Recommendation
- A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models
- Towards Robust and Accurate Stability Estimation of Local Surrogate Models in Text-based Explainable AI
- Mapping the Mind of an Instruction-based Image Editing using SMILE
- Simple Is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation
- Explainability of Point Cloud Neural Networks Using SMILE: Statistical Model-Agnostic Interpretability with Local Explanations
- Graph Retrieval-Augmented Generation: A Survey
- HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction
- From Local to Global: A Graph RAG Approach to Query-Focused Summarization
- Using Stratified Sampling to Improve LIME Image Explanations
- Beyond Pixels: Enhancing LIME with Hierarchical Features and Segmentation Foundation Models
- Rowen: Adaptive Retrieval-Augmented Generation for Hallucination Mitigation in LLMs
- Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models
- Explaining black boxes with a SMILE: Statistical Model-agnostic Interpretability with Local Explanations
- A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
- Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs
- Unifying Large Language Models and Knowledge Graphs: A Roadmap
- Tree of Thoughts: Deliberate Problem Solving with Large Language Models
- SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
- GPT-4 Technical Report
- A Comprehensive Survey on Automatic Knowledge Graph Construction
- A Comprehensive Survey on Automatic Knowledge Graph Construction
- Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability
- Measuring and Narrowing the Compositionality Gap in Language Models
- Towards Faithful Model Explanation in NLP: A Survey
- Atlas: Few-shot Learning with Retrieval Augmented Language Models
- Teaching Models to Express Their Uncertainty in Words
- Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
- STaR: Bootstrapping Reasoning With Reasoning
- Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Survey of Hallucination in Natural Language Generation
- BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
- WebGPT: Browser-assisted question-answering with human feedback
- LexGLUE: A Benchmark Dataset for Legal Language Understanding in English
- TruthfulQA: Measuring How Models Mimic Human Falsehoods
- On the Opportunities and Risks of Foundation Models
- Complex Knowledge Base Question Answering: A Survey
- Extending LIME for Business Process Automation
- A Survey on Complex Knowledge Base Question Answering: Methods, Challenges and Solutions
- QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering
- BayLIME: Bayesian Local Interpretable Model-Agnostic Explanations
- Open-Domain Question Answering Goes Conversational via Question Rewriting
- Reliable Post hoc Explanations: Modeling Uncertainty in Explainability
- A Survey on Complex Question Answering over Knowledge Base: Recent Advances and Challenges
- OptiLIME: Optimized LIME Explanations for Diagnostic Computer Algorithms
- Language Models are Few-Shot Learners
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
- Evaluating and Aggregating Feature-based Model Explanations
- Towards Faithfully Interpretable NLP Systems: How should we define and\n evaluate faithfulness?
- GraphLIME: Local Interpretable Model Explanations for Graph Neural Networks
- Explaining the Explainer: A First Theoretical Analysis of LIME
- Reasoning Over Paths via Knowledge Base Completion
- Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI
- ALIME: Autoencoder Based Approach for Local Interpretability
- Deep Graph Library: A Graph-Centric, Highly-Performant Package for Graph Neural Networks
- The many Shapley values for model explanation
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
- Retrieve, Read, Rerank: Towards End-to-End Multi-Document Reading Comprehension
- The Curious Case of Neural Text Degeneration
- PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Text
- Relation-Aware Graph Attention Network for Visual Question Answering
- Metrics for Explainable AI: Challenges and Prospects
- Techniques for Interpretable Machine Learning
- Model Agnostic Supervised Local Explanations
- Defining Locality for Surrogates in Post-hoc Interpretablity
- Know What You Don't Know: Unanswerable Questions for SQuAD
- Consistent Individualized Feature Attribution for Tree Ensembles
- A Survey Of Methods For Explaining Black Box Models
- A Survey of Methods for Explaining Black Box Models
- Learning to Paraphrase for Question Answering
- DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning
- A Unified Approach to Interpreting Model Predictions
- Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
- "Why Should I Trust You?": Explaining the Predictions of Any Classifier
- Efficient Estimation of Word Representations in Vector Space
- GraphArena: Evaluating and Exploring Large Language Models on Graph Computation
- Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph
Related