Discriminative Unsupervised Feature Learning with Exemplar Convolutional\n Neural Networks

2014/06/26 by Alexey Dosovitskiy, Philipp Fischer, Dosovitskiy, Alexey +7 · 6 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.1406.6909

openalex publication_date 2014/06/26 · openalex created_date 2022/10/03 · openalex updated_date 2026/07/28

Abstract

Deep convolutional networks have proven to be very successful in learning\ntask specific features that allow for unprecedented performance on various\ncomputer vision tasks. Training of such networks follows mostly the supervised\nlearning paradigm, where sufficiently many input-output pairs are required for\ntraining. Acquisition of large training sets is one of the key challenges, when\napproaching a new task. In this paper, we aim for generic feature learning and\npresent an approach for training a convolutional network using only unlabeled\ndata. To this end, we train the network to discriminate between a set of\nsurrogate classes. Each surrogate class is formed by applying a variety of\ntransformations to a randomly sampled 'seed' image patch. In contrast to\nsupervised network training, the resulting feature representation is not class\nspecific. It rather provides robustness to the transformations that have been\napplied during training. This generic feature representation allows for\nclassification results that outperform the state of the art for unsupervised\nlearning on several popular datasets (STL-10, CIFAR-10, Caltech-101,\nCaltech-256). While such generic features cannot compete with class specific\nfeatures from supervised training on a classification task, we show that they\nare advantageous on geometric matching problems, where they also outperform the\nSIFT descriptor.\n

Citations

• Backpropagation Applied to Handwritten Zip Code Recognition
• Deeply-Supervised Nets
• Caffe: Convolutional Architecture for Fast Feature Embedding
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Unsupervised Feature Learning by Cross-Level Instance-Group Discrimination
• Don't Wait, Just Weight: Improving Unsupervised Representations by Learning Goal-Driven Instance Weights
• DisCo: Remedy Self-supervised Learning on Lightweight Models with Distilled Contrastive Learning
• Franca: Nested Matryoshka Clustering for Scalable Visual Representation Learning
• Improving Context-Based Meta-Reinforcement Learning with Self-Supervised Trajectory Contrastive Learning
• FROST: Faster and more Robust One-shot Semi-supervised Training

Related

• Unsupervised feature learning by augmenting single images
• Deep Convolutional Neural Networks as Generic Feature Extractors
• Unsupervised Deep Learning by Neighbourhood Discovery
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Unsupervised Learning by Predicting Noise