Adversarial Feature Distribution Alignment for Semi-Supervised Learning

2019/12/22 by Mayer, Christoph, Paul, Matthieu, Timofte, Radu · 1 citation
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.1912.10428

Abstract

Training deep neural networks with only a few labeled samples can lead to overfitting. This is problematic in semi-supervised learning where only a few labeled samples are available. In this paper, we show that a consequence of overfitting in SSL is feature distribution misalignment between labeled and unlabeled samples. Hence, we propose a new feature distribution alignment method. Our method is particularly effective when using only a small amount of labeled samples. We test our method on CIFAR10 and SVHN. On SVHN we achieve a test error of 3.88% (250 labeled samples) and 3.39% (1000 labeled samples) which is close to the fully supervised model 2.89% (73k labeled samples). In comparison, the current SOTA achieves only 4.29% and 3.74%. Finally, we provide a theoretical insight why feature distribution alignment occurs and show that our method reduces it.

Cited by

• ST-DAI: Single-shot 2.5D Spatial Transcriptomics with Intra-Sample Domain Adaptive Imputation for Cost-efficient 3D Reconstruction

Related

• Semi-Supervised Learning by Augmented Distribution Alignment
• Revisiting Deep Semi-supervised Learning: An Empirical Distribution Alignment Framework and Its Generalization Bound
• Limitations of Post-Hoc Feature Alignment for Robustness
• Learning Compact Features via In-Training Representation Alignment
• Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning