Semi-Supervised Learning with IPM-based GANs: an Empirical Study

2017/12/07 by Tom Sercu, Sercu, Tom, Youssef Mroueh +1
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.1712.02505

openalex publication_date 2017/12/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present an empirical investigation of a recent class of Generative Adversarial Networks (GANs) using Integral Probability Metrics (IPM) and their performance for semi-supervised learning. IPM-based GANs like Wasserstein GAN, Fisher GAN and Sobolev GAN have desirable properties in terms of theoretical understanding, training stability, and a meaningful loss. In this work we investigate how the design of the critic (or discriminator) influences the performance in semi-supervised learning. We distill three key take-aways which are important for good SSL performance: (1) the K+1 formulation, (2) avoiding batch normalization in the critic and (3) avoiding gradient penalty constraints on the classification layer.

Citations

• On integral probability metrics, ϕ-divergences and binary classification
• McGan: Mean and Covariance Feature Matching GAN
• Improved Training of Wasserstein GANs
• Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• Good Semi-supervised Learning that Requires a Bad GAN
• Temporal Ensembling for Semi-Supervised Learning
• Learning Multiple Layers of Features from Tiny Images

Related

• How Does GAN-based Semi-supervised Learning Work?
• Good Semi-supervised Learning that Requires a Bad GAN
• Semi-supervised learning based on generative adversarial network: a comparison between good GAN and bad GAN approach
• Demystifying MMD GANs
• Fisher GAN