Making deep neural networks right for the right scientific reasons by\n interacting with their explanations

2020/01/15 by Patrick Schramowski, Schramowski, Patrick, Wolfgang Stammer +13 · 10 citations
Agricultural and Biological Sciences · Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications #Smart Agriculture and AI

paper · pdf · doi:10.48550/arxiv.2001.05371

openalex publication_date 2020/01/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Deep neural networks have shown excellent performances in many real-world\napplications. Unfortunately, they may show "Clever Hans"-like behavior --\nmaking use of confounding factors within datasets -- to achieve high\nperformance. In this work, we introduce the novel learning setting of\n"explanatory interactive learning" (XIL) and illustrate its benefits on a plant\nphenotyping research task. XIL adds the scientist into the training loop such\nthat she interactively revises the original model via providing feedback on its\nexplanations. Our experimental results demonstrate that XIL can help avoiding\nClever Hans moments in machine learning and encourages (or discourages, if\nappropriate) trust into the underlying model.\n

Cited by

• Consistent Evidence, Robust Recognition: Faithful Attribution Regularization under Geometric Transformations
• LVLM-Aided Alignment of Task-Specific Vision Models
• Human Cognitive Biases in Explanation-Based Interaction: The Case of Within and Between Session Order Effect
• Synthesizing Visual Concepts as Vision-Language Programs
• Hybrid Explanation-Guided Learning for Transformer-Based Chest X-Ray Diagnosis
• Discriminative Feature Feedback with General Teacher Classes
• Debugging Concept Bottleneck Models through Removal and Retraining
• AIM: Amending Inherent Interpretability via Self-Supervised Masking
• Towards trustworthy AI in materials mechanics through domain-guided attention
• Predictive modeling, pattern recognition, and spatiotemporal representations of plant growth in simulated and controlled environments: A comprehensive review

Related

• One Explanation Does Not Fit XIL
• Explanatory Learning: Beyond Empiricism in Neural Networks
• Toward Machine-Guided, Human-Initiated Explanatory Interactive Learning
• Explaining Explanations: An Overview of Interpretability of Machine\n Learning
• Explaining Deep Neural Networks by Leveraging Intrinsic Methods