Stabilizing Training of Generative Adversarial Networks through Regularization

2017/05/25 by Kevin A. Roth, Kevin Roth, Roth, Kevin +7 · 1 voice · 11 citations
Computer Science · Mathematics · Physics and Astronomy · #Computational Physics and Python Applications #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Model Reduction and Neural Networks #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1705.09367

openalex publication_date 2017/05/25 · arxiv published 2017/05/25 · arxiv updated 2017/11/07 · openalex created_date 2019/06/27 · openalex updated_date 2026/07/28

Abstract

Deep generative models based on Generative Adversarial Networks (GANs) have demonstrated impressive sample quality but in order to work they require a careful choice of architecture, parameter initialization, and selection of hyper-parameters. This fragility is in part due to a dimensional mismatch or non-overlapping support between the model distribution and the data distribution, causing their density ratio and the associated f-divergence to be undefined. We overcome this fundamental limitation and propose a new regularization approach with low computational cost that yields a stable GAN training procedure. We demonstrate the effectiveness of this regularizer across several architectures trained on common benchmark image generation tasks. Our regularization turns GAN models into reliable building blocks for deep learning.

Citations

• LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop
• Adam: A Method for Stochastic Optimization
• On integral probability metrics, ϕ-divergences and binary classification
• f-GAN: Training Generative Neural Samplers using Variational Divergence\n Minimization
• Mode Regularized Generative Adversarial Networks
• Improved Training of Wasserstein GANs
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Experts-Guided Unbalanced Optimal Transport for ISP Learning from Unpaired and/or Paired Data
• SpriteHand: Real-Time Versatile Hand-Object Interaction with Autoregressive Video Generation
• Adversarial Flow Models
• MoGAN: Improving Motion Quality in Video Diffusion via Few-Step Motion Adversarial Post-Training
• Distribution Matching Distillation Meets Reinforcement Learning
• Beyond Data Scarcity Optimizing R3GAN for Medical Image Generation from Small Datasets
• Alleviation of Gradient Exploding in GANs: Fake Can Be Real
• Towards the Automatic Anime Characters Creation with Generative Adversarial Networks
• Wasserstein-2 Generative Networks
• Random Matrix Theory Proves that Deep Learning Representations of\n GAN-data Behave as Gaussian Mixtures
• Bridging GANs and Bayesian Neural Networks via Partial Stochasticity

Discussions

• Stabilizing Training of Generative Adversarial Networks Through Regularization [hn, 3 points, 0 comments]

Related

• Regularizing Generative Adversarial Networks under Limited Data
• A Large-Scale Study on Regularization and Normalization in GANs
• Consistency Regularization for Generative Adversarial Networks
• Regularization Methods for Generative Adversarial Networks: An Overview of Recent Studies
• Mode Regularized Generative Adversarial Networks