Towards Faithfully Interpretable NLP Systems: How should we define and\n evaluate faithfulness?

2020/04/07 by Alon Jacovi, Yoav Goldberg, Jacovi, Alon +1 · 62 citations
Computer Science · #Adversarial Robustness in Machine Learning #Computation and Language (cs.CL) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2004.03685

openalex publication_date 2020/04/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

With the growing popularity of deep-learning based NLP models, comes a need\nfor interpretable systems. But what is interpretability, and what constitutes a\nhigh-quality interpretation? In this opinion piece we reflect on the current\nstate of interpretability evaluation research. We call for more clearly\ndifferentiating between different desired criteria an interpretation should\nsatisfy, and focus on the faithfulness criteria. We survey the literature with\nrespect to faithfulness evaluation, and arrange the current approaches around\nthree assumptions, providing an explicit form to how faithfulness is "defined"\nby the community. We provide concrete guidelines on how evaluation of\ninterpretation methods should and should not be conducted. Finally, we claim\nthat the current binary definition for faithfulness sets a potentially\nunrealistic bar for being considered faithful. We call for discarding the\nbinary notion of faithfulness in favor of a more graded one, which we believe\nwill be of greater practical utility.\n

Cited by

• Is Chain-of-Thought Really Not Explainability? Chain-of-Thought Can Be Faithful without Hint Verbalization
• Logic Sketch Prompting (LSP): A Deterministic and Interpretable Prompting Method
• LEX-EC: A Lexical Evidence-Channel Audit Framework for Zero-Shot LLM Personality Classification in Black-Box Settings
• Disentangling the Interpretive and Predictive Roles of LIWC: Controlled Substitution in Depression-Related Classification
• State over Tokens: Characterizing the Role of Reasoning Tokens
• Back to the Baseline: Examining Baseline Effects on Explainability Metrics
• What Triggers my Model? Contrastive Explanations Inform Gender Choices by Translation Models
• Chain-of-Image Generation: Toward Monitorable and Controllable Image Generation
• A Geometric Unification of Concept Learning with Concept Cones
• Investigating Training and Generalization in Faithful Self-Explanations of Large Language Models
• MASE: Interpretable NLP Models via Model-Agnostic Saliency Estimation
• Training and Evaluation of Guideline-Based Medical Reasoning in LLMs
• A Framework for Causal Concept-based Model Explanations
• REFLEX: Self-Refining Explainable Fact-Checking via Disentangling Truth into Style and Substance
• Progressive Localisation in Localist LLMs
• Principled Design of Interpretable Automated Scoring for Large-Scale Educational Assessments
• CID: Measuring Feature Importance Through Counterfactual Distributions
• FaithAct: Faithfulness Planning and Acting in MLLMs
• T-FIX: Text-Based Explanations with Features Interpretable to eXperts
• Before the Clinic: Transparent and Operable Design Principles for Healthcare AI
• Thinking About Thinking: Evaluating Reasoning in Post-Trained Language Models
• Robustness and trustworthiness in AI: a no-go result from formal epistemology
• On the Faithfulness of Visual Thinking: Measurement and Enhancement
• Framework for Machine Evaluation of Reasoning Completeness in Large Language Models For Classification Tasks
• Robust or Suggestible? Exploring Non-Clinical Induction in LLM Drug-Safety Decisions
• Learnable Game-theoretic Policy Optimization for Data-centric Self-explanation Rationalization
• Attribution Graphs and Causal Probing for Mechanistic Discovery and Bias Repair in Multimodal Generative Learning
• Making Power Explicable in AI: Analyzing, Understanding, and Redirecting Power to Operationalize Ethics in AI Technical Practice
• Sparsely gated tiny linear experts
• iBERT: Interpretable Style Embeddings via Sense Decomposition
• From Explainability to Action: A Generative Operational Framework for Integrating XAI in Clinical Mental Health Screening
• GraphGhost: Tracing Structures Behind Large Language Models
• FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning
• Evaluation Framework for Highlight Explanations of Context Utilisation in Language Models
• Activation-Deactivation: A General Framework for Robust Post-hoc Explainable AI
• DecepChain: Inducing Deceptive Reasoning in Large Language Models
• An Experimental Study on Generating Plausible Textual Explanations for Video Summarization
• The Unheard Alternative: Contrastive Explanations for Speech-to-Text Models
• Walk the Talk: Is Your Log-based Software Reliability Maintenance System Really Reliable?
• From Reasoning to Answer: Empirical, Attention-Based and Mechanistic Insights into Distilled DeepSeek R1 Models
• Explanation-Driven Counterfactual Testing for Faithfulness in Vision-Language Model Explanations
• Protocode: Prototype-Driven Interpretability for Code Generation in LLMs
• Explaining multimodal LLMs via intra-modal token interactions
• Bridging Fairness and Explainability: Can Input-Based Explanations Promote Fairness in Hate Speech Detection?
• Context Parametrization with Compositional Adapters
• Towards Transparent AI: A Survey on Explainable Language Models
• Position, Not Provenance: Separating Reasoning Mediation from Sycophancy in Medical Vision-Language Models
• GRPO++: Enhancing Dermatological Reasoning under Low Resource Settings
• nDNA -- the Semantic Helix of Artificial Cognition
• [Re] Improving Interpretation Faithfulness for Vision Transformers
• Do Activation Verbalization Methods Convey Privileged Information?
• A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models
• Explainable Knowledge Graph Retrieval-Augmented Generation (KG-RAG) with KG-SMILE
• MedOmni-45°: A Safety-Performance Benchmark for Reasoning-Oriented LLMs in Medicine
• Inducing Faithfulness in Structured Reasoning via Counterfactual Sensitivity
• RAG-PRISM: A Personalized, Rapid, and Immersive Skill Mastery Framework with Adaptive Retrieval-Augmented Tutoring
• When Explainability Meets Privacy: An Investigation at the Intersection of Post-hoc Explainability and Differential Privacy in the Context of Natural Language Processing
• X-Node: Self-Explanation is All We Need
• LLMs for Resource Allocation: A Participatory Budgeting Approach to Inferring Preferences
• NeuroSync: Intent-Aware Code-Based Problem Solving via Direct LLM Understanding Modification
• TeSent: A Benchmark Dataset for Fairness-aware Explainable Sentiment Classification in Telugu
• TT-XAI: Trustworthy Clinical Text Explanations via Keyword Distillation and LLM Reasoning

Related

• New Faithfulness-Centric Interpretability Paradigms for Natural Language Processing
• On the Sensitivity and Stability of Model Interpretations in NLP
• Are Interpretations Fairly Evaluated? A Definition Driven Pipeline for Post-Hoc Interpretability
• A Comparative Study of Faithfulness Metrics for Model Interpretability Methods
• Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyond