PAC-Bayesian Generalization Bounds for Adversarial Generative Models

2023/02/17 by Sokhna Diarra Mbacke, Mbacke, Sokhna Diarra, Florence Clerc +3 · 2 citations
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.2302.08942

openalex publication_date 2023/02/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/01

Abstract

We extend PAC-Bayesian theory to generative models and develop generalization bounds for models based on the Wasserstein distance and the total variation distance. Our first result on the Wasserstein distance assumes the instance space is bounded, while our second result takes advantage of dimensionality reduction. Our results naturally apply to Wasserstein GANs and Energy-Based GANs, and our bounds provide new training objectives for these two. Although our work is mainly theoretical, we perform numerical experiments showing non-vacuous generalization bounds for Wasserstein GANs on synthetic datasets.

Cited by

• Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis
• Deep Actor-Critics with Tight Risk Certificates

Related

• On the estimation of the Wasserstein distance in generative models
• Generative Modeling using the Sliced Wasserstein Distance
• A Bayesian Non-parametric Approach to Generative Models: Integrating Variational Autoencoder and Generative Adversarial Networks using Wasserstein and Maximum Mean Discrepancy
• A Wasserstein perspective of Vanilla GANs
• Learning Distributions by Generative Adversarial Networks: Approximation and Generalization