2020/10/19 by Jason Phang, Jungkyu Park, Phang, Jason +3 · 1 citation
Computer Science · #Adversarial Robustness in Machine Learning #Computer Vision and Pattern Recognition (cs.CV) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning and Data Classification
paper · pdf · doi:10.48550/arxiv.2010.09750
openalex publication_date 2020/10/19 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28
Saliency maps that identify the most informative regions of an image for a\nclassifier are valuable for model interpretability. A common approach to\ncreating saliency maps involves generating input masks that mask out portions\nof an image to maximally deteriorate classification performance, or mask in an\nimage to preserve classification performance. Many variants of this approach\nhave been proposed in the literature, such as counterfactual generation and\noptimizing over a Gumbel-Softmax distribution. Using a general formulation of\nmasking-based saliency methods, we conduct an extensive evaluation study of a\nnumber of recently proposed variants to understand which elements of these\nmethods meaningfully improve performance. Surprisingly, we find that a\nwell-tuned, relatively simple formulation of a masking-based saliency model\noutperforms many more complex approaches. We find that the most important\ningredients for high quality saliency map generation are (1) using both\nmasked-in and masked-out objectives and (2) training the classifier alongside\nthe masking model. Strikingly, we show that a masking model can be trained with\nas few as 10 examples per class and still generate saliency maps with only a\n0.7-point increase in localization error.\n