S4L: Self-Supervised Semi-Supervised Learning

2019/10/01 by Xiaohua Zhai, Avital Oliver, Alexander Kolesnikov +1 · 4 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #Advanced Image and Video Retrieval Techniques #Multimodal Machine Learning Applications

paper · doi:10.1109/iccv.2019.00156

openalex publication_date 2019/10/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29

Abstract

This work tackles the problem of semi-supervised learning of image classifiers. Our main insight is that the field of semi-supervised learning can benefit from the quickly advancing field of self-supervised visual representation learning. Unifying these two approaches, we propose the framework of self-supervised semi-supervised learning (S4L) and use it to derive two novel semi-supervised image classification methods. We demonstrate the effectiveness of these methods in comparison to both carefully tuned baselines, and existing semi-supervised learning methods. We then show that S4L and existing semi-supervised methods can be jointly trained, yielding a new state-of-the-art result on semi-supervised ILSVRC-2012 with 10% of labels.

Citations

• Unsupervised Visual Representation Learning by Context Prediction
• Semi-Supervised Learning with Ladder Networks
• ImageNet Large Scale Visual Recognition Challenge
• Improved Techniques for Training GANs
• Variational Autoencoder for Deep Learning of Images, Labels and Captions
• In Defense of the Triplet Loss for Person Re-Identification
• Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
• Robustness via Retrying: Closed-Loop Robotic Manipulation with Self-Supervised Learning
• Deep Metric Transfer for Label Propagation with Limited Annotated Data
• MixMatch: A Holistic Approach to Semi-Supervised Learning
• Data-Efficient Image Recognition with Contrastive Predictive Coding
• Time-Contrastive Networks: Self-Supervised Learning from Video
• Unsupervised Data Augmentation for Consistency Training
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Semi-Supervised Class Discovery
• A Unified Mixture-View Framework for Unsupervised Representation Learning
• Weakly Semi-supervised Whole Slide Image Classification by Two-level Cross Consistency Supervision
• Self-supervised learning [wikipedia]

Related

• S4L: Self-Supervised Semi-Supervised Learning
• Color-S⁴L: Self-supervised Semi-supervised Learning with Image Colorization
• Semi-Supervised and Unsupervised Deep Visual Learning: A Survey
• A Probabilistic Model Behind Self-Supervised Learning
• A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends