Wasserstein GANs Work Because They Fail (to Approximate the Wasserstein Distance)

2021/03/02 by Jan Stanczuk, Stanczuk, Jan, Christian Etmann +5 · 4 citations
Computer Science · #Generative Adversarial Networks and Image Synthesis #Adversarial Robustness in Machine Learning #Advanced Neural Network Applications

paper · pdf · doi:10.48550/arxiv.2103.01678

Abstract

Wasserstein GANs are based on the idea of minimising the Wasserstein distance between a real and a generated distribution. We provide an in-depth mathematical analysis of differences between the theoretical setup and the reality of training Wasserstein GANs. In this work, we gather both theoretical and empirical evidence that the WGAN loss is not a meaningful approximation of the Wasserstein distance. Moreover, we argue that the Wasserstein distance is not even a desirable loss function for deep generative models, and conclude that the success of Wasserstein GANs can in truth be attributed to a failure to approximate the Wasserstein distance.

Cited by

• Explainable Learning Based Regularization of Inverse Problems
• Physics-Informed Design of Input Convex Neural Networks for Consistency Optimal Transport Flow Matching
• NGGAN: Noise Generation GAN Based on the Practical Measurement Dataset for Narrowband Powerline Communications
• Slicing Wasserstein Over Wasserstein Via Functional Optimal Transport

Related

• On the estimation of the Wasserstein distance in generative models
• (q,p)-Wasserstein GANs: Comparing Ground Metrics for Wasserstein GANs
• Wasserstein GAN
• A Wasserstein perspective of Vanilla GANs
• From GAN to WGAN