Towards a Unified Evaluation of Explanation Methods without Ground Truth

2019/11/20 by Hao Zhang, Jiayi Chen, Zhang, Hao +5 · 1 citation
Computer Science · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.1911.09017

openalex publication_date 2019/11/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper proposes a set of criteria to evaluate the objectiveness of explanation methods of neural networks, which is crucial for the development of explainable AI, but it also presents significant challenges. The core challenge is that people usually cannot obtain ground-truth explanations of the neural network. To this end, we design four metrics to evaluate explanation results without ground-truth explanations. Our metrics can be broadly applied to nine benchmark methods of interpreting neural networks, which provides new insights of explanation methods.

Citations

• Understanding Neural Networks Through Deep Visualization
• The Pascal Visual Object Classes (VOC) Challenge
• Gradient-based learning applied to document recognition
• ImageNet classification with deep convolutional neural networks
• Harnessing Deep Neural Networks with Logic Rules
• Understanding Black-box Predictions via Influence Functions
• Learning how to explain neural networks: PatternNet and PatternAttribution
• Sanity Checks for Saliency Maps
• Evaluating Explanation Without Ground Truth in Interpretable Machine\n Learning
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• On a Method to Measure Supervised Multiclass Model’s Interpretability: Application to Degradation Diagnosis (Short Paper)
• Benchmarking Attribution Methods with Relative Feature Importance
• Learning Multiple Layers of Features from Tiny Images

Cited by

• A Survey on Interpretability in Visual Recognition

Related

• Do Explanations Explain? Model Knows Best
• Towards Benchmarking Explainable Artificial Intelligence Methods
• Evaluating Neuron Explanations: A Unified Framework with Sanity Checks
• Explanation Methods in Deep Learning: Users, Values, Concerns and\n Challenges
• Aggregating explanation methods for stable and robust explainability