Semi-Supervised Learning by Label Gradient Alignment

2019/02/06 by Jacob Jackson, John Schulman, Jackson, Jacob +1
Computer Science · Mathematics · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1902.02336

arxiv created 2019/02/06 · openalex publication_date 2019/02/06 · arxiv updated 2019/02/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present label gradient alignment, a novel algorithm for semi-supervised learning which imputes labels for the unlabeled data and trains on the imputed labels. We define a semantically meaningful distance metric on the input space by mapping a point (x, y) to the gradient of the model at (x, y). We then formulate an optimization problem whose objective is to minimize the distance between the labeled and the unlabeled data in this space, and we solve it by gradient descent on the imputed labels. We evaluate label gradient alignment using the standardized architecture introduced by Oliver et al. (2018) and demonstrate state-of-the-art accuracy in semi-supervised CIFAR-10 classification.

Citations

• Semi-Supervised Learning with Ladder Networks
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Understanding Black-box Predictions via Influence Functions
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Gradient Similarity: An Explainable Approach to Detect Adversarial Attacks against Deep Learning
• Adapting Auxiliary Losses Using Gradient Similarity

Related

• Semi-Supervised Learning by Augmented Distribution Alignment
• FlatMatch: Bridging Labeled Data and Unlabeled Data with Cross-Sharpness for Semi-Supervised Learning
• CLS: Cross Labeling Supervision for Semi-Supervised Learning
• Revisiting Deep Semi-supervised Learning: An Empirical Distribution Alignment Framework and Its Generalization Bound
• Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning