FROST: Faster and more Robust One-shot Semi-supervised Training

2020/11/18 by Helena E. Liu, Leslie N. Smith, Liu, Helena E. +1
Computer Science · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Multimodal Machine Learning Applications #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2011.09471

openalex publication_date 2020/11/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recent advances in one-shot semi-supervised learning have lowered the barrier for deep learning of new applications. However, the state-of-the-art for semi-supervised learning is slow to train and the performance is sensitive to the choices of the labeled data and hyper-parameter values. In this paper, we present a one-shot semi-supervised learning method that trains up to an order of magnitude faster and is more robust than state-of-the-art methods. Specifically, we show that by combining semi-supervised learning with a one-stage, single network version of self-training, our FROST methodology trains faster and is more robust to choices for the labeled samples and changes in hyper-parameters. Our experiments demonstrate FROST's capability to perform well when the composition of the unlabeled data is unknown; that is when the unlabeled data contain unequal numbers of each class and can contain out-of-distribution examples that don't belong to any of the training classes. High performance, speed of training, and insensitivity to hyper-parameters make FROST the most practical method for one-shot semi-supervised training. Our code is available at https://github.com/HelenaELiu/FROST.

Citations

• Wide Residual Networks
• Improved Regularization of Convolutional Neural Networks with Cutout
• Discriminative Unsupervised Feature Learning with Exemplar Convolutional\n Neural Networks
• Unsupervised Data Augmentation for Consistency Training
• Self-training with Noisy Student improves ImageNet classification
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence
• A Simple Framework for Contrastive Learning of Visual Representations
• Improved Baselines with Momentum Contrastive Learning
• Prototypical Contrastive Learning of Unsupervised Representations
• Bootstrap your own latent: A new approach to self-supervised Learning
• Big Self-Supervised Models are Strong Semi-Supervised Learners
• Learning Multiple Layers of Features from Tiny Images

Related

• Building One-Shot Semi-supervised (BOSS) Learning up to Fully Supervised Performance
• Empirical Perspectives on One-Shot Semi-supervised Learning
• DoubleMatch: Improving Semi-Supervised Learning with Self-Supervision
• FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning
• Learning to Self-Train for Semi-Supervised Few-Shot Classification