Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?

2021/05/22 by Jie Ren, Zhanpeng Zhou, Ren, Jie +5 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Bayesian Modeling and Causal Inference #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2105.10719

openalex publication_date 2021/05/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Masking some input variables of a deep neural network (DNN) and computing output changes on the masked input sample represent a typical way to compute attributions of input variables in the sample. People usually mask an input variable using its baseline value. However, there is no theory to examine whether baseline value faithfully represents the absence of an input variable, i.e., removing all signals from the input variable. Fortunately, recent studies show that the inference score of a DNN can be strictly disentangled into a set of causal patterns (or concepts) encoded by the DNN. Therefore, we propose to use causal patterns to examine the faithfulness of baseline values. More crucially, it is proven that causal patterns can be explained as the elementary rationale of the Shapley value. Furthermore, we propose a method to learn optimal baseline values, and experimental results have demonstrated its effectiveness.

Citations

• Estimators of Relative Importance in Linear Regression Based on Variance Decomposition
• Gradient-based learning applied to document recognition
• ImageNet classification with deep convolutional neural networks
• Consistent Individualized Feature Attribution for Tree Ensembles
• Local Rule-Based Explanations of Black Box Decision Systems
• Explaining individual predictions when features are dependent: More accurate approximations to Shapley values
• On a Method to Measure Supervised Multiclass Model’s Interpretability: Application to Degradation Diagnosis (Short Paper)
• Problems with Shapley-value-based explanations as feature importance\n measures
• Learning Multiple Layers of Features from Tiny Images

Cited by

• INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Sampling

Related

• HarsanyiNet: Computing Accurate Shapley Values in a Single Forward Propagation
• Suboptimal Shapley Value Explanations
• Interpreting Multivariate Shapley Interactions in DNNs
• Causal Shapley Values: Exploiting Causal Knowledge to Explain Individual Predictions of Complex Models
• Defining and Quantifying the Emergence of Sparse Concepts in DNNs