Minimizing False-Positive Attributions in Explanations of Non-Linear Models

2025/05/16 by Anders Gjølbye, Stefan Haufe, Gjølbye, Anders +3
Computer Science · Neuroscience · #Adversarial Robustness in Machine Learning #EEG and Brain-Computer Interfaces #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.2505.11210

openalex publication_date 2025/05/16 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Suppressor variables can influence model predictions without being dependent on the target outcome, and they pose a significant challenge for Explainable AI (XAI) methods. These variables may cause false-positive feature attributions, undermining the utility of explanations. Although effective remedies exist for linear models, their extension to non-linear models and instance-based explanations has remained limited. We introduce PatternLocal, a novel XAI technique that addresses this gap. PatternLocal begins with a locally linear surrogate, e.g., LIME, KernelSHAP, or gradient-based methods, and transforms the resulting discriminative model weights into a generative representation, thereby suppressing the influence of suppressor variables while preserving local fidelity. In extensive hyperparameter optimization on the XAI-TRIS benchmark, PatternLocal consistently outperformed other XAI methods and reduced false-positive attributions when explaining non-linear tasks, thereby enabling more reliable and actionable insights. We further evaluate PatternLocal on an EEG motor imagery dataset, demonstrating physiologically plausible explanations.

Citations

• Explainable AI needs formal notions of explanation correctness
• GLIME: General, Stable and Local LIME Explanation
• XAI-TRIS: Non-linear image benchmarks to quantify false positive post-hoc attribution of feature importance
• Benchmarking the Influence of Pre-training on Explanation Performance in MR Image Classification
• Theoretical Behavior of XAI Methods in the Presence of Suppressor Variables
• XAI for Transformers: Better Explanations through Conservative Propagation
• Scrutinizing XAI using linear ground-truth data with suppressor variables
• A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
• Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
• SmoothGrad: removing noise by adding noise
• A Unified Approach to Interpreting Model Predictions
• Learning how to explain neural networks: PatternNet and PatternAttribution
• Deep learning with convolutional neural networks for EEG decoding and visualization
• Axiomatic Attribution for Deep Networks
• Striving for Simplicity: The All Convolutional Net
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• Visualizing and Understanding Convolutional Networks
• Discrete, 3D distributed, linear imaging methods of electric neuronal activity. Part 1: exact, zero error localization
• BCI2000: A General-Purpose Brain-Computer Interface (BCI) System
• A well-conditioned estimator for large-dimensional covariance matrices
• A Revised Definition for Suppressor Variables: a Guide To Their Identification and Interpretation

Related

• XAI-TRIS: Non-linear image benchmarks to quantify false positive post-hoc attribution of feature importance
• Gradient based Feature Attribution in Explainable AI: A Technical Review
• Respect the model: Fine-grained and Robust Explanation with Sharing Ratio Decomposition
• Exploring Local Explanations of Nonlinear Models Using Animated Linear Projections
• GLIME: A new graphical methodology for interpretable model-agnostic explanations