Wasserstein Convergence Guarantees for a General Class of Score-Based Generative Models

2023/11/18 by Xuefeng Gao, Gao, Xuefeng, Hoang Minh Nguyen +3 · 17 citations
Computer Science · Medicine · #Advanced Neuroimaging Techniques and Applications #FOS: Computer and information sciences #FOS: Mathematics #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Medical Image Segmentation Techniques #Probability (math.PR)

paper · pdf · doi:10.48550/arxiv.2311.11003

openalex publication_date 2023/11/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Score-based generative models (SGMs) is a recent class of deep generative models with state-of-the-art performance in many applications. In this paper, we establish convergence guarantees for a general class of SGMs in 2-Wasserstein distance, assuming accurate score estimates and smooth log-concave data distribution. We specialize our result to several concrete SGMs with specific choices of forward processes modelled by stochastic differential equations, and obtain an upper bound on the iteration complexity for each model, which demonstrates the impacts of different choices of the forward processes. We also provide a lower bound when the data distribution is Gaussian. Numerically, we experiment SGMs with different forward processes, some of which are newly proposed in this paper, for unconditional image generation on CIFAR-10. We find that the experimental results are in good agreement with our theoretical predictions on the iteration complexity, and the models with our newly proposed forward processes can outperform existing models.

Cited by

• Pathway to O(√(d)) Complexity bound under Wasserstein metric of flow-based models
• Dimension-free error estimate for diffusion model and optimal scheduling
• Sublinear iterations can suffice even for DDPMs
• Wasserstein Convergence of Critically Damped Langevin Diffusions
• Optimal Convergence Analysis of DDPM for General Distributions
• Error analysis of a compositional score-based algorithm for simulation-based inference
• Assessing the Quality of Denoising Diffusion Models in Wasserstein Distance: Noisy Score and Optimal Bounds
• Understanding Sampler Stochasticity in Training Diffusion Models for RLHF
• Factor-Based Conditional Diffusion Model for Contextual Portfolio Optimization
• Discrete Diffusion Models: Novel Analysis and New Sampler Guarantees
• A Statistical Benchmark for Diffusion Posterior Sampling Algorithms
• Diffusion Generative Models Meet Compressed Sensing, with Applications to Imaging and Finance
• Distribution estimation via Flow Matching with Lipschitz guarantees
• Convergence of Deterministic and Stochastic Diffusion-Model Samplers: A Simple Analysis in Wasserstein Distance
• Dynamic data generation and dynamic portfolio selection: an application of a score-based diffusion model
• Wasserstein Convergence of Score-based Generative Models under Semiconvexity and Discontinuous Gradients
• Understanding Generalization in Diffusion Distillation via Probability Flow Distance
• Score Accuracy Along the Forward Diffusion Does Not Certify Numerical Stability in Diffusion Sampling

Related

• Wasserstein Convergence of Score-based Generative Models under Semiconvexity and Discontinuous Gradients
• Convergence of score-based generative modeling for general data distributions
• Generalization bounds for score-based generative models: a synthetic proof
• An analysis of the noise schedule for score-based generative models
• Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions