Improving Generalization via Scalable Neighborhood Component Analysis

2018/08/14 by Zhirong Wu, Alexei A. Efros, Wu, Zhirong +3 · 5 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.1808.04699

openalex publication_date 2018/08/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Current major approaches to visual recognition follow an end-to-end formulation that classifies an input image into one of the pre-determined set of semantic categories. Parametric softmax classifiers are a common choice for such a closed world with fixed categories, especially when big labeled data is available during training. However, this becomes problematic for open-set scenarios where new categories are encountered with very few examples for learning a generalizable parametric classifier. We adopt a non-parametric approach for visual recognition by optimizing feature embeddings instead of parametric classifiers. We use a deep neural network to learn the visual feature that preserves the neighborhood structure in the semantic space, based on the Neighborhood Component Analysis (NCA) criterion. Limited by its computational bottlenecks, we devise a mechanism to use augmented memory to scale NCA for large datasets and very deep networks. Our experiments deliver not only remarkable performance on ImageNet classification for such a simple non-parametric method, but most importantly a more generalizable feature representation for sub-category discovery and few-shot recognition.

Citations

• The Pascal Visual Object Classes (VOC) Challenge
• Long Short-Term Memory
• ImageNet Large Scale Visual Recognition Challenge
• Estimating the Support of a High-Dimensional Distribution
• Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups
• What makes ImageNet good for transfer learning?
• Prototypical Networks for Few-shot Learning
• Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
• Sampling Matters in Deep Embedding Learning
• Meta-SGD: Learning to Learn Quickly for Few-Shot Learning
• CosFace: Large Margin Cosine Loss for Deep Face Recognition
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Deep Metric Transfer for Label Propagation with Limited Annotated Data
• Topology-Aware Modeling for Unsupervised Simulation-to-Reality Point Cloud Recognition
• Calibrated neighborhood aware confidence measure for deep metric learning
• Self-supervised Latent Space Optimization with Nebula Variational Coding
• Spectral Feature Transformation for Person Re-identification

Related

• Memory-Based Neighbourhood Embedding for Visual Recognition
• Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination
• Learning and aggregating deep local descriptors for instance-level\n recognition
• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• When Naïve Bayes Nearest Neighbours Meet Convolutional Neural Networks