Improved Training of Wasserstein GANs

2017/03/31 by Ishaan Gulrajani, Faruk Ahmed, Gulrajani, Ishaan +7 · 217 citations
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.1704.00028

openalex publication_date 2017/03/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Generative Adversarial Networks (GANs) are powerful generative models, but suffer from training instability. The recently proposed Wasserstein GAN (WGAN) makes progress toward stable training of GANs, but sometimes can still generate only low-quality samples or fail to converge. We find that these problems are often due to the use of weight clipping in WGAN to enforce a Lipschitz constraint on the critic, which can lead to undesired behavior. We propose an alternative to clipping weights: penalize the norm of gradient of the critic with respect to its input. Our proposed method performs better than standard WGAN and enables stable training of a wide variety of GAN architectures with almost no hyperparameter tuning, including 101-layer ResNets and language models over discrete data. We also achieve high quality generations on CIFAR-10 and LSUN bedrooms.

Citations

• f-GAN: Training Generative Neural Samplers using Variational Divergence\n Minimization
• Improved Techniques for Training GANs
• Instance Normalization: The Missing Ingredient for Fast Stylization
• SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
• Categorical Reparameterization with Gumbel-Softmax
• Conditional Image Synthesis With Auxiliary Classifier GANs
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Meta-information Guided Cross-domain Synergistic Diffusion Model for Low-dose PET Reconstruction
• Advancing Marine Bioacoustics with Deep Generative Models: A Hybrid Augmentation Strategy for Southern Resident Killer Whale Detection
• Flash-CNNCap: Capacitance Extraction via Image Mapping
• Secure and Explainable Fraud Detection in Finance via Hierarchical Multi-source Dataset Distillation
• Regression generation adversarial network based on dual data evaluation strategy for industrial application
• Generative Modeling through Koopman Spectral Analysis: An Operator-Theoretic Perspective
• Improving Cardiac Risk Prediction Using Data Generation Techniques
• MCR-VQGAN: A Scalable and Cost-Effective Tau PET Synthesis Approach for Alzheimer's Disease Imaging
• MFE-GAN: Efficient GAN-based Framework for Document Image Enhancement and Binarization with Multi-scale Feature Extraction
• An intercomparison of generative machine learning methods for downscaling precipitation at fine spatial scales
• Adaptive Sampling for Hydrodynamic Stability
• Towards Physically-Based Sky-Modeling For Image Based Lighting
• Enhancing Node-Level Graph Domain Adaptation by Alleviating Local Dependency
• Differentiable Energy-Based Regularization in GANs: A Simulator-Based Exploration of VQE-Inspired Auxiliary Losses
• SVG-T2I: Scaling Up Text-to-Image Latent Diffusion Model Without Variational Autoencoder
• Causal Attribution of Model Performance Gaps in Medical Imaging Under Distribution Shifts
• A Novel Wasserstein Quaternion Generative Adversarial Network for Color Image Generation
• CADE: Continual Weakly-supervised Video Anomaly Detection with Ensembles
• Impugan: Learning Conditional Generative Models for Robust Data Imputation
• Self-Supervised AI-Generated Image Detection: A Camera Metadata Perspective
• Semantic-Guided Two-Stage GAN for Face Inpainting with Hybrid Perceptual Encoding
• Plug-and-Play Image Restoration with Flow Matching: A Continuous Viewpoint
• 2-Shots in the Dark: Low-Light Denoising with Minimal Data Acquisition
• Optimizing Distributional Geometry Alignment with Optimal Transport for Generative Dataset Distillation
• Invited to Develop: Institutional Belonging and the Counterfactual Architecture of Development
• Adversarial Flow Models
• Waveform-Based Probabilistic Seismic Hazard Analysis Using Ground-Motion Generative Models
• Long-Term Alzheimers Disease Prediction: A Novel Image Generation Method Using Temporal Parameter Estimation with Normal Inverse Gamma Distribution on Uneven Time Series
• Probabilistic Wildfire Spread Prediction Using an Autoregressive Conditional Generative Adversarial Network
• HVAdam: A Full-Dimension Adaptive Optimizer
• Supervise Less, See More: Training-free Nuclear Instance Segmentation with Prototype-Guided Prompting
• Targeted Manipulation: Slope-Based Attacks on Financial Time-Series Data
• Efficient Transferable Optimal Transport via Min-Sliced Transport Plans
• VAOT: Vessel-Aware Optimal Transport for Retinal Fundus Enhancement
• How to Train Your Latent Control Barrier Function: Smooth Safety Filtering Under Hard-to-Model Constraints
• GANGR: GAN-Assisted Scalable and Efficient Global Routing Parallelization
• Denoising weak lensing mass maps with diffusion model and generative adversarial network
• Explainable AI for Diabetic Retinopathy Detection Using Deep Learning with Attention Mechanisms and Fuzzy Logic-Based Interpretability
• Synergizing Deconfounding and Temporal Generalization For Time-series Counterfactual Outcome Estimation
• Adversarial Learning-Based Radio Map Reconstruction for Fingerprinting Localization
• Improving conditional generative adversarial networks for inverse design of plasmonic structures
• Adaptive Symmetrization of the KL Divergence
• Gradient-Guided Exploration of Generative Model's Latent Space for Controlled Iris Image Augmentations
• Equivariant Sampling for Improving Diffusion Model-based Image Restoration
• Incremental Generation is Necessary and Sufficient for Universality in Flow-Based Modelling
• DKDS: A Benchmark Dataset of Degraded Kuzushiji Documents with Seals for Detection and Binarization
• HybridGuard: Enhancing Minority-Class Intrusion Detection in Dew-Enabled Edge-of-Things Networks
• Enhancing Multimodal Misinformation Detection by Replaying the Whole Story from Image Modality Perspective
• An Interdisciplinary and Cross-Task Review on Missing Data Imputation
• BI-DCGAN: A Theoretically Grounded Bayesian Framework for Efficient and Diverse GANs
• Generative Artificial Intelligence for Air Shower Simulation
• Learning to Manage Investment Portfolios beyond Simple Utility Functions
• BOLT-GAN: Bayes-Optimal Loss for Stable GAN Training
• A Statistician Teaches Deep Learning
• Unsupervised Learning of Solutions to Differential Equations with Generative Adversarial Networks
• Deriving Machine Attention from Human Rationales
• Finding Mixed Nash Equilibria of Generative Adversarial Networks
• What Does It Take to Detect an AI Agent? Minimal Feature Sets for Behavioral Detection under Browser Automation
• Mode Collapse and Regularity of Optimal Transportation Maps
• Generating the support with extreme value losses
• Deep Convolutional GANs for Car Image Generation
• On the regularization of Wasserstein GANs
• Generative adversarial networks for speech processing: A review
• A Comprehensive Evaluation Framework for Synthetic Trip Data Generation in Public Transport
• Causal-Aware Generative Adversarial Networks with Reinforcement Learning
• Robust and Generalizable Background Subtraction on Images of Calorimeter Jets using Unsupervised Generative Learning
• Logical GANs: Adversarial Learning through Ehrenfeucht Fraisse Games
• Discovering Latent Graphs with GFlowNets for Diverse Conditional Image Generation
• Lead Sheet Generation and Arrangement by Conditional Generative Adversarial Network
• Generative AI in Depth: A Survey of Recent Advances, Model Variants, and Real-World Applications
• Video-to-Video Synthesis
• GAS: Improving Discretization of Diffusion ODEs via Generalized Adversarial Solver
• Quantum Synthetic Data Generation for Industrial Bioprocess Monitoring
• Plasma Shape Control via Zero-shot Generative Reinforcement Learning
• Conditional Synthetic Live and Spoof Fingerprint Generation
• Latent Diffusion Model without Variational Autoencoder
• Expert or not? assessing data quality in offline reinforcement learning
• Deep SPI: Safe Policy Improvement via World Models
• Bounded Information Rate Variational Autoencoders
• Encoder Decoder Generative Adversarial Network Model for Stock Market Prediction
• Automated Defect Detection for Mass-Produced Electronic Components Based on YOLO Object Detection Models
• NGGAN: Noise Generation GAN Based on the Practical Measurement Dataset for Narrowband Powerline Communications
• Extreme value forecasting using relevance-based data augmentation with deep learning models
• Improved Network Robustness with Adversary Critic
• Disassembling Object Representations without Labels
• Adversarial Contrastive Estimation
• Unsupervised Cipher Cracking Using Discrete GANs
• On Discovering Algorithms for Adversarial Imitation Learning
• Population Synthesis using Incomplete Information
• Out-of-Sample Testing for GANs
• Learning Energy-based Variational Latent Prior for VAEs
• CODED-SMOOTHING: Coding Theory Helps Generalization
• YOLO-Based Defect Detection for Metal Sheets
• Unsupervised Detection of Spatiotemporal Anomalies in PMU Data Using Transformer-Based BiGAN
• Noise-Guided Transport for Imitation Learning
• Cycle Diffusion Model for Counterfactual Image Generation
• General-to-Detailed GAN for Infrequent Class Medical Images
• Scalable GANs with Transformers
• An Agent-Based Framework for Automated Higher-Voice Harmony Generation
• Learning to Ball: Composing Policies for Long-Horizon Basketball Moves
• Learning Multi-Skill Legged Locomotion Using Conditional Adversarial Motion Priors
• T2I-Diff: fMRI Signal Generation via Time-Frequency Image Transform and Classifier-Free Denoising Diffusion Models
• Guiding the One-to-one Mapping in CycleGAN via Optimal Transport
• Generating Adversarial Examples with Adversarial Networks
• MMD GAN: Towards Deeper Understanding of Moment Matching Network
• Approximation Analysis of the Entropic Penalty in Quadratic Programming
• Generative Adversarial Networks Applied for Privacy Preservation in Biometric-Based Authentication and Identification
• TIMED: Adversarial and Autoregressive Refinement of Diffusion-Based Time Series Generation
• Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step
• Video-to-Video Translation for Visual Speech Synthesis
• Improved Training with Curriculum GANs
• SPG-Net: Segmentation Prediction and Guidance Network for Image Inpainting
• Adversarially Regularized Autoencoders
• Shilling Recommender Systems by Generating Side-feature-aware Fake User Profiles
• Generating Realistic Sequences of Customer-level Transactions for Retail Datasets
• Exploring Machine Learning Models for Physical Dose Calculation in Carbon Ion Therapy Using Heterogeneous Imaging Data -- A Proof of Concept Study
• HyPlaneHead: Rethinking Tri-plane-like Representations in Full-Head Image Synthesis
• Generative AI Meets Wireless Sensing: Towards Wireless Foundation Model
• RaceGAN: A Framework for Preserving Individuality while Converting Racial Information for Image-to-Image Translation
• Generative Consistency Models for Estimation of Kinetic Parametric Image Posteriors in Total-Body PET
• Bridging the Gap between Label- and Reference-based Synthesis in Multi-attribute Image-to-Image Translation
• Controllable Descendant Face Synthesis
• AREPAS: Anomaly Detection in Fine-Grained Anatomy with Reconstruction-Based Semantic Patch-Scoring
• Learned convex regularizers for inverse problems
• 3D Object Reconstruction from a Single Depth View with Adversarial Learning
• ST-GAN: Spatial Transformer Generative Adversarial Networks for Image Compositing
• Make Identity Unextractable yet Perceptible: Synthesis-Based Privacy Protection for Subject Faces in Photos
• Simulating Sinogram-Domain Motion and Correcting Image-Domain Artifacts Using Deep Learning in HR-pQCT Bone Imaging
• Towards Better Long-range Time Series Forecasting using Generative Adversarial Networks
• On the Effectiveness of Least Squares Generative Adversarial Networks
• Multiscaling in Wasserstein Spaces
• BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN
• Linear Convergence of Gradient Descent for Quadratically Regularized Optimal Transport
• GTA-Crime: A Synthetic Dataset and Generation Framework for Fatal Violence Detection with Adversarial Snippet-Level Domain Adaptation
• The Cramer Distance as a Solution to Biased Wasserstein Gradients
• ArtifactGen: Benchmarking WGAN-GP vs Diffusion for Label-Aware EEG Artifact Synthesis
• Data Augmentation Generative Adversarial Networks
• Physics-informed Value Learner for Offline Goal-Conditioned Reinforcement Learning
• Moment- and Power-Spectrum-Based Gaussianity Regularization for Text-to-Image Models
• InfoVAE: Information Maximizing Variational Autoencoders
• QCA-MolGAN: Quantum Circuit Associative Molecular GAN with Multi-Agent Reinforcement Learning
• Dual-Scale Volume Priors with Wasserstein-Based Consistency for Semi-Supervised Medical Image Segmentation
• Boundary-Seeking Generative Adversarial Networks
• Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks
• Adversarial Binary Coding for Efficient Person Re-identification
• Set Transformer Architectures and Synthetic Data Generation for Flow-Guided Nanoscale Localization
• Adversarial Learning of Label Dependency: A Novel Framework for\n Multi-class Classification
• On the Collapse Errors Induced by the Deterministic Sampler for Diffusion Models
• Unsupervised Detection of Lesions in Brain MRI using constrained adversarial auto-encoders
• Semi-Supervised Bayesian GANs with Log-Signatures for Uncertainty-Aware Credit Card Fraud Detection
• Distributed Gossip-GAN for Low-overhead CSI Feedback Training in FDD mMIMO-OFDM Systems
• The relativistic discriminator: a key element missing from standard GAN
• Double-Constraint Diffusion Model with Nuclear Regularization for Ultra-low-dose PET Reconstruction
• Towards Digital Twins for Optimal Radioembolization
• Quantum enhanced ensemble GANs for anomaly detection in continuous biomanufacturing
• An Optimistic Gradient Tracking Method for Distributed Minimax Optimization
• Normalized Wasserstein Distance for Mixture Distributions with Applications in Adversarial Learning and Domain Adaptation
• TET-GAN: Text Effects Transfer via Stylization and Destylization
• RIGA: Covert and Robust White-Box Watermarking of Deep Neural Networks
• Twin-GAN -- Unpaired Cross-Domain Image Translation with Weight-Sharing GANs
• Quantum latent distributions in deep generative models
• Improving Movement Predictions of Traffic Actors in Bird's-Eye View Models using GANs and Differentiable Trajectory Rasterization
• Distance-informed Neural Processes
• Controllable Single-shot Animation Blending with Temporal Conditioning
• Stabilizing Training of Generative Adversarial Networks through Regularization
• Stability-certified reinforcement learning: A control-theoretic perspective
• Potential and challenges of generative adversarial networks for super-resolution in 4D Flow MRI
• 3D-PhysNet: Learning the Intuitive Physics of Non-Rigid Object Deformations
• Multi-Domain Supervised Contrastive Learning for UAV Radio-Frequency Open-Set Recognition
• Design by adaptive sampling
• Variational Approaches for Auto-Encoding Generative Adversarial Networks
• TequilaGAN: How to easily identify GAN samples
• Domain Translation of a Soft Robotic Arm using Conditional Cycle Generative Adversarial Network
• Conditioning of three-dimensional generative adversarial networks for pore and reservoir-scale models
• RegimeNAS: Regime-Aware Differentiable Architecture Search With Theoretical Guarantees for Financial Trading
• Probabilistic Forecasting Method for Offshore Wind Farm Cluster under Typhoon Conditions: a Score-Based Conditional Diffusion Model
• Hybrid Quantum-Classical Latent Diffusion Models for Medical Image Generation
• Image Inpainting via Generative Multi-column Convolutional Neural Networks
• Generation of Indian Sign Language Letters, Numbers, and Words
• Improving GAN Training via Binarized Representation Entropy (BRE)\n Regularization
• Attacks on State-of-the-Art Face Recognition using Attentional Adversarial Attack Generative Network
• Π-nets: Deep Polynomial Neural Networks
• Boosting Image Outpainting with Semantic Layout Prediction
• Deep Generative Models for Discrete Genotype Simulation
• Generative Inversion for Property-Targeted Materials Design: Application to Shape Memory Alloys
• Undress to Redress: A Training-Free Framework for Virtual Try-On
• Enhancing Privacy in Decentralized Min-Max Optimization: A Differentially Private Approach
• Quantum-Enhanced Generative Adversarial Networks: Comparative Analysis of Classical and Hybrid Quantum-Classical Generative Adversarial Networks
• Towards Useful and Private Synthetic Omics: Community Benchmarking of Generative Models for Transcriptomics Data
• Improved Adversarial Systems for 3D Object Generation and Reconstruction
• Score-Guided Generative Adversarial Networks
• Quantum Reservoir GAN
• CogniPlan: Uncertainty-Guided Path Planning with Conditional Generative Layout Prediction
• Hand-Eye Autonomous Delivery: Learning Humanoid Navigation, Locomotion and Reaching
• DISSECT: Disentangled Simultaneous Explanations via Concept Traversals
• RDDPM: Robust Denoising Diffusion Probabilistic Model for Unsupervised Anomaly Segmentation
• Neural Networks with Orthogonal Jacobian
• Optimal Transport for Rectified Flow Image Editing: Unifying Inversion-Based and Direct Methods
• An empirical study on evaluation metrics of generative adversarial networks
• Parametric Synthesis of Text on Stylized Backgrounds using PGGANs
• Regression Augmentation With Data-Driven Segmentation
• A machine learning approach to tomographic pattern generation and classification of quantum states of light
• A Conditional GAN for Tabular Data Generation with Probabilistic Sampling of Latent Subspaces
• Wind Power Scenario Generation based on the Generalized Dynamic Factor Model and Generative Adversarial Network
• Metric Learning-based Generative Adversarial Network
• Towards Unsupervised Automatic Speech Recognition Trained by Unaligned Speech and Text only
• Large-Scale Optimal Transport via Adversarial Training with Cycle-Consistency
• Simulation-based inference for Precision Neutrino Physics through Neural Monte Carlo tuning
• Adversarial Generation of Natural Language
• Visual Language Models as Zero-Shot Deepfake Detectors
• Towards Privacy-preserving Photorealistic Self-avatars in Mixed Reality
• Quantum generative modeling for financial time series with temporal correlations
• Evaluating Deepfake Detectors in the Wild
• Molecular Attributes Transfer from Non-Parallel Data
• Comparing Generative Adversarial Network Techniques for Image Creation\n and Modification
• Statistical analysis of Wasserstein GANs with applications to time series forecasting
• PanoGAN A Deep Generative Model for Panoramic Dental Radiographs
• Harnessing Diffusion-Yielded Score Priors for Image Restoration

Related

• A Wasserstein GAN model with the total variational regularization
• Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect
• Inferential Wasserstein Generative Adversarial Networks
• On the regularization of Wasserstein GANs
• Adversarial Lipschitz Regularization