Real Time Image Saliency for Black Box Classifiers

2017/05/22 by Piotr Dabkowski, Yarin Gal, Dabkowski, Piotr +1 · 7 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #FOS: Computer and information sciences #Machine Learning (stat.ML) #Visual Attention and Saliency Detection

paper · pdf · doi:10.48550/arxiv.1705.07857

openalex publication_date 2017/05/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this work we develop a fast saliency detection method that can be applied to any differentiable image classifier. We train a masking model to manipulate the scores of the classifier by masking salient parts of the input image. Our model generalises well to unseen images and requires a single forward pass to perform saliency detection, therefore suitable for use in real-time systems. We test our approach on CIFAR-10 and ImageNet datasets and show that the produced saliency maps are easily interpretable, sharp, and free of artifacts. We suggest a new metric for saliency and test our method on the ImageNet object localisation task. We achieve results outperforming other weakly supervised methods.

Citations

• Intriguing properties of neural networks
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• ImageNet Large Scale Visual Recognition Challenge
• Striving for Simplicity: The All Convolutional Net
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• Learning Multiple Layers of Features from Tiny Images

Cited by

• TextCAM: Explaining Class Activation Map with Text
• On the notion of missingness for path attribution explainability methods in medical settings: Guiding the selection of medically meaningful baselines
• Attribution Explanations for Deep Neural Networks: A Theoretical Perspective
• Explainable Artificial Intelligence in Biomedical Image Analysis: A Comprehensive Survey
• Do Protein Transformers Have Biological Intelligence?
• Anomaly Detection in Event-triggered Traffic Time Series via Similarity Learning
• Pixel-level Certified Explanations via Randomized Smoothing

Related

• Hallucinating Saliency Maps for Fine-Grained Image Classification for Limited Data Domains
• A Deep Learning Based Fast Image Saliency Detection Algorithm
• Investigating and Simplifying Masking-based Saliency Methods for Model\n Interpretability
• Saliency Detection Via Similar Image Retrieval
• SalFAU-Net: Saliency Fusion Attention U-Net for Salient Object Detection