ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring

2019/11/21 by David Berthelot, Berthelot, David, Nicholas Carlini +11 · 10 citations
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.1911.09785

openalex publication_date 2019/11/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We improve the recently-proposed "MixMatch" semi-supervised learning algorithm by introducing two new techniques: distribution alignment and augmentation anchoring. Distribution alignment encourages the marginal distribution of predictions on unlabeled data to be close to the marginal distribution of ground-truth labels. Augmentation anchoring feeds multiple strongly augmented versions of an input into the model and encourages each output to be close to the prediction for a weakly-augmented version of the same input. To produce strong augmentations, we propose a variant of AutoAugment which learns the augmentation policy while the model is being trained. Our new algorithm, dubbed ReMixMatch, is significantly more data-efficient than prior work, requiring between 5× and 16× less data to reach the same accuracy. For example, on CIFAR-10 with 250 labeled examples we reach 93.73% accuracy (compared to MixMatch's accuracy of 93.58% with 4,000 examples) and a median accuracy of 84.92% with just four labels per class. We make our code and data open-source at https://github.com/google-research/remixmatch.

Cited by

• CATCHFed: Efficient Unlabeled Data Utilization for Semi-Supervised Federated Learning in Limited Labels Environments
• Semi-Supervised Learning under General Causal Models
• BARL: Bilateral Alignment in Representation and Label Spaces for Semi-Supervised Volumetric Medical Image Segmentation
• SpikeMatch: Semi-Supervised Learning with Temporal Dynamics of Spiking Neural Networks
• SADA: Safe and Adaptive Aggregation of Multiple Black-Box Predictions in Semi-Supervised Learning
• FLARE-SSM: Deep State Space Models with Influence-Balanced Loss for 72-Hour Solar Flare Prediction
• Collaborative Learning of Scattering and Deep Features for SAR Target Recognition with Noisy Labels
• When Is Prior Knowledge Helpful? Exploring the Evaluation and Selection of Unsupervised Pretext Tasks from a Neuro-Symbolic Perspective
• JanusNet: Hierarchical Slice-Block Shuffle and Displacement for Semi-Supervised 3D Multi-Organ Segmentation
• CaliMatch: Adaptive Calibration for Improving Safe Semi-supervised Learning

Related

• Semi-Supervised Learning by Augmented Distribution Alignment
• MixMatch: A Holistic Approach to Semi-Supervised Learning
• Combining MixMatch and Active Learning for Better Accuracy with Fewer Labels
• SequenceMatch: Revisiting the design of weak-strong augmentations for Semi-supervised learning
• FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence