Right for the Right Reasons: Training Differentiable Models by\n Constraining their Explanations

2017/03/10 by Andrew Slavin Ross, Ross, Andrew Slavin, Michael C. Hughes +4 · 1 voice · 37 citations
Computer Science · #Explainable Artificial Intelligence (XAI) #Adversarial Robustness in Machine Learning #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.1703.03717

Abstract

Neural networks are among the most accurate supervised learning methods in\nuse today, but their opacity makes them difficult to trust in critical\napplications, especially when conditions in training differ from those in test.\nRecent work on explanations for black-box models has produced tools (e.g. LIME)\nto show the implicit rules behind predictions, which can help us identify when\nmodels are right for the wrong reasons. However, these methods do not scale to\nexplaining entire datasets and cannot correct the problems they reveal. We\nintroduce a method for efficiently explaining and regularizing differentiable\nmodels by examining and selectively penalizing their input gradients, which\nprovide a normal to the decision boundary. We apply these penalties both based\non expert annotation and in an unsupervised fashion that encourages diverse\nmodels with qualitatively different decision boundaries for the same\nclassification problem. On multiple datasets, we show our approach generates\nfaithful explanations and models that generalize much better when conditions\ndiffer between training and test.\n

Cited by

• Local Additive Feature Attribution: A Mathematical Taxonomy and Reporting Checklist
• The Quadrilateral Loss: Additivity as a Measurable Behavior of Dense Neural Networks
• K-IPO: Kendall-constrained Importance Preserving Oversampling for Imbalanced Tabular Data
• Controlling Implicit Shortcut Reliance in L2 Spoken English Auto-markers
• People Can Accurately Predict Behavior of Complex Algorithms That Are Available, Compact, and Aligned
• Consistent Evidence, Robust Recognition: Faithful Attribution Regularization under Geometric Transformations
• LVLM-Aided Alignment of Task-Specific Vision Models
• Localising Shortcut Learning in Pixel Space via Ordinal Scoring Correlations for Attribution Representations (OSCAR)
• LLMCache: Layer-Wise Caching Strategies for Accelerated Reuse in Transformer Inference
• Hybrid Attribution Priors for Explainable and Robust Model Training
• Measuring the Effect of Background on Classification and Feature Importance in Deep Learning for AV Perception
• Human Cognitive Biases in Explanation-Based Interaction: The Case of Within and Between Session Order Effect
• Learning from Sufficient Rationales: Analysing the Relationship Between Explanation Faithfulness and Token-level Regularisation Strategies
• From Attribution to Action: Jointly ALIGNing Predictions and Explanations
• Seed-Induced Uniqueness in Transformer Models: Subspace Alignment Governs Subliminal Transfer
• Understanding Hardness of Vision-Language Compositionality from A Token-level Causal Lens
• FaCT: Faithful Concept Traces for Explaining Neural Network Decisions
• Learning from Interval Targets
• Classifying polish in use-wear analysis with convolutional neural networks
• MedCLM: Learning to Localize and Reason via a CoT-Curriculum in Medical Vision-Language Models
• Batch-CAM: Introduction to better reasoning in convolutional deep learning models
• Benchmarking explainable AI methods for toxicophore detection and toxicity prediction
• Accuracy-Robustness Trade Off via Spiking Neural Network Gradient Sparsity Trail
• Causally-Enhanced Reinforcement Policy Optimization
• Learning to Look: Cognitive Attention Alignment with Vision-Language Models
• Debugging Concept Bottleneck Models through Removal and Retraining
• On Aligning Prediction Models with Clinical Experiential Learning: A Prostate Cancer Case Study
• AIM: Amending Inherent Interpretability via Self-Supervised Masking
• TRACEALIGN -- Tracing the Drift: Attributing Alignment Failures to Training-Time Belief Sources in LLMs
• Inherently Faithful Attention Maps for Vision Transformers
• Neural Concept Verifier: Scaling Prover-Verifier Games via Concept Encodings
• Machine Learning from Explanations
• Transferring Visual Explainability of Self-Explaining Models to Prediction-Only Models without Additional Training
• Model Guidance via Robust Feature Attribution
• Attribution-Guided Pruning for Insight and Control: Circuit Discovery and Targeted Correction in Small-scale LLMs
• Efficient Text-Attributed Graph Learning through Selective Annotation and Graph Alignment
• TIMING: Temporality-Aware Integrated Gradients for Time Series Explanation

Discussions

• Training Differentiable Models by Constraining Their Explanations [hn, 36 points, 5 comments]

Related

• Training Machine Learning Models by Regularizing their Explanations
• Learning with Explanation Constraints
• Learning explanations that are hard to vary
• Building Reliable Explanations of Unreliable Neural Networks: Locally Smoothing Perspective of Model Interpretation
• Regularizing Black-box Models for Improved Interpretability