2019/08/20 by Gamaleldin F. Elsayed, Simon Kornblith, Elsayed, Gamaleldin F. +3 · 4 citations
Computer Science · Mathematics · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Multimodal Machine Learning Applications #cs.CV #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.1908.07644
33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada
openalex publication_date 2019/08/20 · arxiv created 2019/12/07 · arxiv updated 2019/12/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Although deep convolutional neural networks achieve state-of-the-art performance across nearly all image classification tasks, their decisions are difficult to interpret. One approach that offers some level of interpretability by design is hard attention, which uses only relevant portions of the image. However, training hard attention models with only class label supervision is challenging, and hard attention has proved difficult to scale to complex datasets. Here, we propose a novel hard attention model, which we term Saccader. Key to Saccader is a pretraining step that requires only class labels and provides initial attention locations for policy gradient optimization. Our best models narrow the gap to common ImageNet baselines, achieving 75% top-1 and 91% top-5 while attending to less than one-third of the image.