Discretized Integrated Gradients for Explaining Language Models

2021/08/31 by Soumya Sanyal, Xiang Ren, Sanyal, Soumya +1 · 5 citations
Computer Science · #Computation and Language (cs.CL) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning in Healthcare #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2108.13654

openalex publication_date 2021/08/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

As a prominent attribution-based explanation algorithm, Integrated Gradients (IG) is widely adopted due to its desirable explanation axioms and the ease of gradient computation. It measures feature importance by averaging the model's output gradient interpolated along a straight-line path in the input data space. However, such straight-line interpolated points are not representative of text data due to the inherent discreteness of the word embedding space. This questions the faithfulness of the gradients computed at the interpolated points and consequently, the quality of the generated explanations. Here we propose Discretized Integrated Gradients (DIG), which allows effective attribution along non-linear interpolation paths. We develop two interpolation strategies for the discrete word embedding space that generates interpolation points that lie close to actual words in the embedding space, yielding more faithful gradient computation. We demonstrate the effectiveness of DIG over IG through experimental and human evaluations on multiple sentiment classification datasets. We provide the source code of DIG to encourage reproducible research.

Citations

• Understanding Neural Networks through Representation Erasure
• Towards A Rigorous Science of Interpretable Machine Learning
• Axiomatic Attribution for Deep Networks
• On a Method to Measure Supervised Multiclass Model’s Interpretability: Application to Degradation Diagnosis (Short Paper)
• Generalized Integrated Gradients: A practical method for explaining diverse ensembles
• LLM-Supported Manufacturing Mapping Generation
• Towards Hierarchical Importance Attribution: Explaining Compositional Semantics for Neural Sequence Models

Cited by

• HEMERA: A Human-Explainable Transformer Model for Estimating Lung Cancer Risk using GWAS Data
• Evaluation Framework for Highlight Explanations of Context Utilisation in Language Models
• Graph-based Integrated Gradients for Explaining Graph Neural Networks
• Toward the Explainability of Protein Language Models
• XAI-Units: Benchmarking Explainability Methods with Unit Tests

Related

• Uniform Discretized Integrated Gradients: An effective attribution based method for explaining large language models
• Sequential Integrated Gradients: a simple but effective method for explaining language models
• On Gradient-like Explanation under a Black-box Setting: When Black-box Explanations Become as Good as White-box
• Four Axiomatic Characterizations of the Integrated Gradients Attribution Method
• Explanation Bias is a Product: Revealing the Hidden Lexical and Position Preferences in Post-Hoc Feature Attribution