2020/01/15 by Paola Cascante-Bonilla, Cascante-Bonilla, Paola, Fuwen Tan +5
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Data Stream Mining Techniques #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification
paper · pdf · doi:10.48550/arxiv.2001.06001
openalex publication_date 2020/01/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In this paper we revisit the idea of pseudo-labeling in the context of\nsemi-supervised learning where a learning algorithm has access to a small set\nof labeled samples and a large set of unlabeled samples. Pseudo-labeling works\nby applying pseudo-labels to samples in the unlabeled set by using a model\ntrained on the combination of the labeled samples and any previously\npseudo-labeled samples, and iteratively repeating this process in a\nself-training cycle. Current methods seem to have abandoned this approach in\nfavor of consistency regularization methods that train models under a\ncombination of different styles of self-supervised losses on the unlabeled\nsamples and standard supervised losses on the labeled samples. We empirically\ndemonstrate that pseudo-labeling can in fact be competitive with the\nstate-of-the-art, while being more resilient to out-of-distribution samples in\nthe unlabeled set. We identify two key factors that allow pseudo-labeling to\nachieve such remarkable results (1) applying curriculum learning principles and\n(2) avoiding concept drift by restarting model parameters before each\nself-training cycle. We obtain 94.91% accuracy on CIFAR-10 using only 4,000\nlabeled samples, and 68.87% top-1 accuracy on Imagenet-ILSVRC using only 10% of\nthe labeled samples. The code is available at\nhttps://github.com/uvavision/Curriculum-Labeling\n