Unsupervised Representation Learning by Predicting Image Rotations

2018/03/21 by Spyros Gidaris, Gidaris, Spyros, Praveer Singh +3 · 54 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.1803.07728

openalex publication_date 2018/03/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Over the last years, deep convolutional neural networks (ConvNets) have transformed the field of computer vision thanks to their unparalleled capacity to learn high level semantic image features. However, in order to successfully learn those features, they usually require massive amounts of manually labeled data, which is both expensive and impractical to scale. Therefore, unsupervised semantic feature learning, i.e., learning without requiring manual annotation effort, is of crucial importance in order to successfully harvest the vast amount of visual data that are available today. In our work we propose to learn image features by training ConvNets to recognize the 2d rotation that is applied to the image that it gets as input. We demonstrate both qualitatively and quantitatively that this apparently simple task actually provides a very powerful supervisory signal for semantic feature learning. We exhaustively evaluate our method in various unsupervised feature learning benchmarks and we exhibit in all of them state-of-the-art performance. Specifically, our results on those benchmarks demonstrate dramatic improvements w.r.t. prior state-of-the-art approaches in unsupervised representation learning and thus significantly close the gap with supervised feature learning. For instance, in PASCAL VOC 2007 detection task our unsupervised pre-trained AlexNet model achieves the state-of-the-art (among unsupervised methods) mAP of 54.4% that is only 2.4 points lower from the supervised case. We get similarly striking results when we transfer our unsupervised learned features on various other tasks, such as ImageNet classification, PASCAL classification, PASCAL segmentation, and CIFAR-10 classification. The code and models of our paper will be published on: https://github.com/gidariss/FeatureLearningRotNet .

Citations

• The Pascal Visual Object Classes (VOC) Challenge
• Gradient-based learning applied to document recognition
• ImageNet Large Scale Visual Recognition Challenge
• Adversarial Feature Learning
• Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
• Colorization as a Proxy Task for Visual Understanding
• Learning Representations for Automatic Colorization
• Learning to See by Moving
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Beam-Response Contrastive Learning for Transmitter-Side MIMO CSI Representation
• Task-Aligned Self-Supervised Learning for Medical Image Analysis: A Task-Oriented Review with Practical Design Guidelines
• Next-Embedding Prediction Makes Strong Vision Learners
• PuzzleCraft: Exploration-Aware Curriculum Learning for Puzzle-Based RLVR in VLMs
• PSMamba: Progressive Self-supervised Vision Mamba for Plant Disease Recognition
• Reducing Label Dependency in Human Activity Recognition with Wearables: From Supervised Learning to Novel Weakly Self-Supervised Approaches
• Supervised Contrastive Frame Aggregation for Video Representation Learning
• OWL: Unsupervised 3D Object Detection by Occupancy Guided Warm-up and Large Model Priors Reasoning
• Self-Supervised AI-Generated Image Detection: A Camera Metadata Perspective
• Domain Feature Collapse: Implications for Out-of-Distribution Detection and Solutions
• A Probabilistic Framework for Temporal Distribution Generalization in Industry-Scale Recommender Systems
• RNN as Linear Transformer: A Closer Investigation into Representational Potentials of Visual Mamba Models
• ViSS-R1: Self-Supervised Reinforcement Video Reasoning
• Out-of-Context Misinformation Detection via Variational Domain-Invariant Learning with Test-Time Training
• Twist and Compute: The Cost of Pose in 3D Generative Diffusion
• Learning from the Right Patches: A Two-Stage Wavelet-Driven Masked Autoencoder for Histopathology Representation Learning
• An Augmentation Overlap Theory of Contrastive Learning
• Diffusion Models are Robust Pretrainers
• C-LEAD: Contrastive Learning for Enhanced Adversarial Defense
• Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learning
• MoME: Mixture of Visual Language Medical Experts for Medical Imaging Segmentation
• Eigenfunction Extraction for Ordered Representation Learning
• SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoning
• Revising Second Order Terms in Deep Animation Video Coding
• Learning Without Augmenting: Unsupervised Time Series Representation Learning via Frame Projections
• Understanding Self-supervised Contrastive Learning through Supervised Objectives
• Conditional Representation Learning for Customized Tasks
• Unsupervised Transformer Pre-Training for Images: Self-Distillation, Mean Teachers, and Random Crops
• REALIGN: Regularized Procedure Alignment with Matching Video Embeddings via Partial Gromov-Wasserstein Optimal Transport
• GenView++: Unifying Adaptive Generative Augmentation and Quality-Driven Supervision for Contrastive Representation Learning
• Category Discovery: An Open-World Perspective
• SSVIF: Self-Supervised Segmentation-Oriented Visible and Infrared Image Fusion
• On the Status of Foundation Models for SAR Imagery
• Dual-supervised Asymmetric Co-training for Semi-supervised Medical Domain Generalization
• DiSSECT: Structuring Transfer-Ready Medical Image Representations through Discrete Self-Supervision
• Global Minimizers of Sigmoid Contrastive Loss
• Latent Zoning Network: A Unified Principle for Generative Modeling, Representation Learning, and Classification
• Leveraging Geometric Visual Illusions as Perceptual Inductive Biases for Vision Models
• Beyond Instance Consistency: Investigating View Diversity in Self-supervised Learning
• Semantic Concentration for Self-Supervised Dense Representations Learning
• Enhancing 3D Medical Image Understanding with Pretraining Aided by 2D Multimodal Large Language Models
• MAE-SAM2: Mask Autoencoder-Enhanced SAM2 for Clinical Retinal Vascular Leakage Segmentation
• Self-Supervised Cross-Encoder for Neurodegenerative Disease Diagnosis
• Adapt in the Wild: Test-Time Entropy Minimization with Sharpness and Feature Regularization
• PLanTS: Periodicity-aware Latent-state Representation Learning for Multivariate Time Series
• Self-Validated Learning for Particle Separation: A Correctness-Based Self-Training Framework Without Human Labels
• ADVMEM: Adversarial Memory Initialization for Realistic Test-Time Adaptation via Tracklet-Based Benchmarking
• What Can We Learn from Harry Potter? An Exploratory Study of Visual Representation Learning from Atypical Videos
• Multi-Level Knowledge Distillation and Dynamic Self-Supervised Learning for Continual Learning
• VELVET-Med: Vision and Efficient Language Pre-training for Volumetric Imaging Tasks in Medicine
• Mask & Match: Learning to Recognize Handwritten Math with Self-Supervised Attention
• Self-Supervised YOLO: Leveraging Contrastive Learning for Label-Efficient Object Detection
• Sample-Aware Test-Time Adaptation for Medical Image-to-Image Translation
• Bi-Level Optimization for Self-Supervised AI-Generated Face Detection

Related

• Unsupervised Visual Representation Learning by Context Prediction
• Unsupervised Representation Learning by Discovering Reliable Image Relations
• Towards Unsupervised Representation Learning: Learning, Evaluating and Transferring Visual Representations
• Image Enhanced Rotation Prediction for Self-Supervised Learning
• Unsupervised feature learning by augmenting single images