Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions

2024/09/27 by Iskander Azangulov, Azangulov, Iskander, George Deligiannidis +3 · 12 citations
Computer Science · #Advanced Mathematical Modeling in Engineering #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Statistics Theory (math.ST)

paper · pdf · doi:10.48550/arxiv.2409.18804

openalex publication_date 2024/09/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/30

Abstract

Denoising Diffusion Probabilistic Models (DDPM) are powerful state-of-the-art methods used to generate synthetic data from high-dimensional data distributions and are widely used for image, audio, and video generation as well as many more applications in science and beyond. The manifold hypothesis states that high-dimensional data often lie on lower-dimensional manifolds within the ambient space, and is widely believed to hold in provided examples. While recent results have provided invaluable insight into how diffusion models adapt to the manifold hypothesis, they do not capture the great empirical success of these models, making this a very fruitful research direction. In this work, we study DDPMs under the manifold hypothesis and prove that they achieve rates independent of the ambient dimension in terms of score learning. In terms of sampling complexity, we obtain rates independent of the ambient dimension w.r.t. the Kullback-Leibler divergence, and O(√(D)) w.r.t. the Wasserstein distance. We do this by developing a new framework connecting diffusion models to the well-studied theory of extrema of Gaussian Processes.

Cited by

• Simultaneous Approximation of the Score Function and Its Derivatives by Deep Neural Networks
• Approximation Capabilities of Feedforward Neural Networks with GELU Activations
• Dimension-free error estimate for diffusion model and optimal scheduling
• Denoising growth complexity: Data geometry and certified schedules for diffusion sampling
• Model-free filtering in high dimensions via projection and score-based diffusions
• Optimal Stopping in Latent Diffusion Models
• Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptive
• Optimal estimation of a factorizable density using diffusion models with ReLU neural networks
• When Scores Learn Geometry: Rate Separations under the Manifold Hypothesis
• Diffusion and Flow-based Copulas: Forgetting and Remembering Dependencies
• Distribution estimation via Flow Matching with Lipschitz guarantees
• Convergence of Deterministic and Stochastic Diffusion-Model Samplers: A Simple Analysis in Wasserstein Distance

Related

• Convergence of denoising diffusion models under the manifold hypothesis
• Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models
• Linear Convergence of Diffusion Models Under the Manifold Hypothesis
• A Note on the Convergence of Denoising Diffusion Probabilistic Models
• Resolving Memorization in Empirical Diffusion Model for Manifold Data in High-Dimensional Spaces