Investigating the influence of noise and distractors on the interpretation of neural networks

2016/11/22 by Pieter-Jan Kindermans, Kristof T. Schütt, Kindermans, Pieter-Jan +5 · 4 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · #Adversarial Robustness in Machine Learning #Cell Image Analysis Techniques #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.1611.07270

openalex publication_date 2016/11/22 · openalex created_date 2016/11/30 · openalex updated_date 2026/07/28

Abstract

Understanding neural networks is becoming increasingly important. Over the last few years different types of visualisation and explanation methods have been proposed. However, none of them explicitly considered the behaviour in the presence of noise and distracting elements. In this work, we will show how noise and distracting dimensions can influence the result of an explanation model. This gives a new theoretical insights to aid selection of the most appropriate explanation model within the deep-Taylor decomposition framework.

Cited by

• Can LLMs Faithfully Explain Themselves in Low-Resource Languages? A Case Study on Emotion Detection in Persian
• Bridging Fairness and Explainability: Can Input-Based Explanations Promote Fairness in Hate Speech Detection?
• Explaining First Impressions: Modeling, Recognizing, and Explaining Apparent Personality from Videos
• Attribution Mask: Filtering Out Irrelevant Features By Recursively Focusing Attention on Inputs of DNNs

Related

• Explanations can be manipulated and geometry is to blame
• Local Explanation Methods for Deep Neural Networks Lack Sensitivity to Parameter Values
• Do Explanations Explain? Model Knows Best
• Explaining Deep Neural Networks
• Neuron-based explanations of neural networks sacrifice completeness and interpretability