2018/06/19 by Vitali Petsiuk, Petsiuk, Vitali, Abir Das +3 · 208 citations
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Artificial intelligence #Artificial neural network #Benchmark (surveying) #Black box #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Contrast (vision) #Data mining #Deep learning #Deep neural networks #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine learning #Metric (unit) #Object (grammar) #Pattern recognition (psychology) #Pixel #Process (computing) #White box #cs.CV
paper · pdf · doi:10.48550/arxiv.1806.07421
published in arXiv (Cornell University), 151 (Cornell University)
openalex publication_date 2018/06/19 · arxiv created 2018/09/25 · arxiv updated 2018/09/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Deep neural networks are being used increasingly to automate data analysis and decision making, yet their decision-making process is largely unclear and is difficult to explain to the end users. In this paper, we address the problem of Explainable AI for deep neural networks that take images as input and output a class probability. We propose an approach called RISE that generates an importance map indicating how salient each pixel is for the model's prediction. In contrast to white-box approaches that estimate pixel importance using gradients or other internal network state, RISE works on black-box models. It estimates importance empirically by probing the model with randomly masked versions of the input image and obtaining the corresponding outputs. We compare our approach to state-of-the-art importance extraction methods using both an automatic deletion/insertion metric and a pointing metric based on human-annotated object segments. Extensive experiments on several benchmark datasets show that our approach matches or exceeds the performance of other methods, including white-box approaches. Project page: http://cs-people.bu.edu/vpetsiuk/rise/