Geometrical Insights for Implicit Generative Modeling

2017/12/21 by Bottou, Leon, Arjovsky, Martin, Lopez-Paz, David +1
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.1712.07822

Abstract

Learning algorithms for implicit generative models can optimize a variety of criteria that measure how the data distribution differs from the implicit model distribution, including the Wasserstein distance, the Energy distance, and the Maximum Mean Discrepancy criterion. A careful look at the geometries induced by these distances on the space of probability measures reveals interesting differences. In particular, we can establish surprising approximate global convergence guarantees for the 1-Wasserstein distance,even when the parametric generator has a nonconvex parametrization.

Related

• Asymptotic Guarantees for Generative Modeling Based on the Smooth Wasserstein Distance
• Asymptotic Guarantees for Learning Generative Models with the Sliced-Wasserstein Distance
• Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections
• On Deep Generative Models for Approximation and Estimation of Distributions on Manifolds
• A Characteristic Function Approach to Deep Implicit Generative Modeling