Regularization Methods for Generative Adversarial Networks: An Overview of Recent Studies

2020/05/19 by Minhyeok Lee, Lee, Minhyeok, Junhee Seok +1 · 1 citation
Computer Science · Engineering · #Advanced Image Processing Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Generative Adversarial Networks and Image Synthesis #Image and Signal Denoising Methods #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #Neural and Evolutionary Computing (cs.NE) #cs.CV #cs.LG #cs.NE #eess.IV #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2005.09165

arxiv created 2020/05/19 · openalex publication_date 2020/05/19 · arxiv updated 2020/05/20 · openalex created_date 2020/05/29 · openalex updated_date 2026/07/28

Abstract

Despite its short history, Generative Adversarial Network (GAN) has been extensively studied and used for various tasks, including its original purpose, i.e., synthetic sample generation. However, applying GAN to different data types with diverse neural network architectures has been hindered by its limitation in training, where the model easily diverges. Such a notorious training of GANs is well known and has been addressed in numerous studies. Consequently, in order to make the training of GAN stable, numerous regularization methods have been proposed in recent years. This paper reviews the regularization methods that have been recently introduced, most of which have been published in the last three years. Specifically, we focus on general methods that can be commonly used regardless of neural network architectures. To explore the latest research trends in the regularization for GANs, the methods are classified into several groups by their operation principles, and the differences between the methods are analyzed. Furthermore, to provide practical knowledge of using these methods, we investigate popular methods that have been frequently employed in state-of-the-art GANs. In addition, we discuss the limitations in existing methods and propose future research directions.

Citations

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Lagrange Multipliers and Optimality
• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Energy-based Generative Adversarial Network
• Mode Regularized Generative Adversarial Networks
• BEGAN: Boundary Equilibrium Generative Adversarial Networks
• Spectral Norm Regularization for Improving the Generalizability of Deep Learning
• Variational Approaches for Auto-Encoding Generative Adversarial Networks
• Hierarchical Implicit Models and Likelihood-Free Variational Inference
• On the regularization of Wasserstein GANs
• On Convergence and Stability of GANs
• MaskGAN: Better Text Generation via Filling in the______
• Deep Facial Expression Recognition: A Survey
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks
• Lipschitz Generative Adversarial Nets
• Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks
• Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect
• Score-Guided Generative Adversarial Networks

Cited by

• Estimation with Uncertainty via Conditional Generative Adversarial Networks

Related

• Stabilizing Training of Generative Adversarial Networks through Regularization
• A Large-Scale Study on Regularization and Normalization in GANs
• Consistency Regularization for Generative Adversarial Networks
• Regularizing Generative Adversarial Networks under Limited Data
• Mode Regularized Generative Adversarial Networks