Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet

2019/03/20 by Wieland Brendel, Matthias Bethge, Brendel, Wieland +1 · 1 voice · 17 citations
Computer Science · Engineering · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #Industrial Vision Systems and Defect Detection #cs.CV #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1904.00760

openalex publication_date 2019/03/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Deep Neural Networks (DNNs) excel on many complex perceptual tasks but it has proven notoriously difficult to understand how they reach their decisions. We here introduce a high-performance DNN architecture on ImageNet whose decisions are considerably easier to explain. Our model, a simple variant of the ResNet-50 architecture called BagNet, classifies an image based on the occurrences of small local image features without taking into account their spatial ordering. This strategy is closely related to the bag-of-feature (BoF) models popular before the onset of deep learning and reaches a surprisingly high accuracy on ImageNet (87.6% top-5 for 33 x 33 px features and Alexnet performance for 17 x 17 px features). The constraint on local features makes it straight-forward to analyse how exactly each part of the image influences the classification. Furthermore, the BagNets behave similar to state-of-the art deep neural networks such as VGG-16, ResNet-152 or DenseNet-169 in terms of feature sensitivity, error distribution and interactions between image parts. This suggests that the improvements of DNNs over previous bag-of-feature classifiers in the last few years is mostly achieved by better fine-tuning rather than by qualitatively different decision strategies.

Cited by

• Enhancing Interpretability for Vision Models via Shapley Value Optimization
• On the Brittleness of CLIP Text Encoders
• Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in\n Visual Question Answering
• Scaling Non-Parametric Sampling with Representation
• Restricted Receptive Fields for Face Verification
• Fast and accurate explanations of distance-based classifiers by uncovering latent explanatory structures
• Shortcut learning in deep neural networks
• Decomposing and Revising What Language Models Generate
• Learning local and global prototypes with optimal transport for unsupervised anomaly detection and localization
• Shortcut Learning in Generalist Robot Policies: The Role of Dataset Diversity and Fragmentation
• Multimodal Attention-Aware Fusion for Diagnosing Distal Myopathy: Evaluating Model Interpretability and Clinician Trust
• SIDE: Sparse Information Disentanglement for Explainable Artificial Intelligence
• Attri-Net: A Globally and Locally Inherently Interpretable Model for Multi-Label Classification Using Class-Specific Counterfactuals
• Feature-Enhanced TResNet for Fine-Grained Food Image Classification
• Transferring Styles for Reduced Texture Bias and Improved Robustness in Semantic Segmentation Networks
• VERITAS: Verification and Explanation of Realness in Images for Transparency in AI Systems
• Thousand-Brains Systems: Sensorimotor Intelligence for Rapid, Robust Learning and Inference

Discussions

• I'm not being dismissive, but I do wonder what competence these models can be rigorously shown to have. This came after almost a decade of furious hand-waving about the amazing capabilities of image c [bsky, 5 points, 1 comments]

Related

• Glance and Focus: a Dynamic Approach to Reducing Spatial Redundancy in Image Classification
• Rethinking the Image Feature Biases Exhibited by Deep CNN Models
• Wider or Deeper: Revisiting the ResNet Model for Visual Recognition
• ImageNot: A contrast with ImageNet preserves model rankings
• Latent Model Ensemble with Auto-localization