Negative sampling in semi-supervised learning

2019/11/12 by John Chen, Chen, John, Vatsal Shah +3 · 1 citation
Computer Science · Medicine · #COVID-19 diagnosis using AI #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.1911.05166

openalex publication_date 2019/11/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce Negative Sampling in Semi-Supervised Learning (NS3L), a simple, fast, easy to tune algorithm for semi-supervised learning (SSL). NS3L is motivated by the success of negative sampling/contrastive estimation. We demonstrate that adding the NS3L loss to state-of-the-art SSL algorithms, such as the Virtual Adversarial Training (VAT), significantly improves upon vanilla VAT and its variant, VAT with Entropy Minimization. By adding the NS3L loss to MixMatch, the current state-of-the-art approach on semi-supervised tasks, we observe significant improvements over vanilla MixMatch. We conduct extensive experiments on the CIFAR10, CIFAR100, SVHN and STL10 benchmark datasets.

Citations

• Adam: A Method for Stochastic Optimization
• ImageNet Large Scale Visual Recognition Challenge
• Distributed Representations of Words and Phrases and their Compositionality
• Generalization error bounds in semi-supervised classification under the cluster assumption
• Wide Residual Networks
• Variational Autoencoder for Deep Learning of Images, Labels and Captions
• Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
• mixup: Beyond Empirical Risk Minimization
• MixMatch: A Holistic Approach to Semi-Supervised Learning
• Unsupervised Data Augmentation for Consistency Training
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Nested Graph Pseudo-Label Refinement for Noisy Label Domain Adaptation Learning

Related

• Boosting Semi-Supervised Learning by Exploiting All Unlabeled Data
• Semantically-Conditioned Negative Samples for Efficient Contrastive Learning
• MixMatch: A Holistic Approach to Semi-Supervised Learning
• DoubleMatch: Improving Semi-Supervised Learning with Self-Supervision
• ADT-SSL: Adaptive Dual-Threshold for Semi-Supervised Learning