On the regularization of Wasserstein GANs

2017/09/26 by Henning Petzka, Asja Fischer, Petzka, Henning +3 · 25 citations
Computer Science · Physics and Astronomy · #Advanced Image Processing Techniques #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Model Reduction and Neural Networks

paper · pdf · doi:10.48550/arxiv.1709.08894

openalex publication_date 2017/09/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Since their invention, generative adversarial networks (GANs) have become a popular approach for learning to model a distribution of real (unlabeled) data. Convergence problems during training are overcome by Wasserstein GANs which minimize the distance between the model and the empirical distribution in terms of a different metric, but thereby introduce a Lipschitz constraint into the optimization problem. A simple way to enforce the Lipschitz constraint on the class of functions, which can be modeled by the neural network, is weight clipping. It was proposed that training can be improved by instead augmenting the loss by a regularization term that penalizes the deviation of the gradient of the critic (as a function of the network's input) from one. We present theoretical arguments why using a weaker regularization term enforcing the Lipschitz constraint is preferable. These arguments are supported by experimental results on toy data sets.

Citations

• Improved Techniques for Training GANs
• Improved Training of Wasserstein GANs

Cited by

• BOLT-GAN: Bayes-Optimal Loss for Stable GAN Training
• What is the Cost of Differential Privacy for Deep Learning-Based Trajectory Generation?
• Composition and decomposition of GANs
• Enhancing Mixup-based Semi-Supervised Learning with Explicit Lipschitz Regularization
• Guiding the One-to-one Mapping in CycleGAN via Optimal Transport
• Uncertainty Estimation Using a Single Deep Deterministic Neural Network
• Conditioning of three-dimensional generative adversarial networks for pore and reservoir-scale models
• General Probabilistic Surface Optimization and Log Density Estimation
• Input Invex Neural Network
• Alleviation of Gradient Exploding in GANs: Fake Can Be Real
• Towards Efficient and Unbiased Implementation of Lipschitz Continuity in GANs
• DeepFlow: History Matching in the Space of Deep Generative Models
• On reproduction of On the regularization of Wasserstein GANs
• Adversarial Computation of Optimal Transport Maps
• Lipschitz Generative Adversarial Nets
• Regularization Methods for Generative Adversarial Networks: An Overview of Recent Studies
• PA-GAN: Progressive Attention Generative Adversarial Network for Facial Attribute Editing
• Exactly Computing the Local Lipschitz Constant of ReLU Networks
• Language Modeling with Generative Adversarial Networks
• Orthogonal Wasserstein GANs
• Banach Wasserstein GAN
• Lipschitz neural networks are dense in the set of all Lipschitz functions
• Recurrent Adversarial Service Times
• Disentangled Recurrent Wasserstein Autoencoder
• A Non-linear Differential CNN-Rendering Module for 3D Data Enhancement

Related

• Adversarial Lipschitz Regularization
• Improved Training of Wasserstein GANs
• A Wasserstein GAN model with the total variational regularization
• How does Lipschitz Regularization Influence GAN Training?
• Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect