Universal Smoothed Score Functions for Generative Modeling

2023/03/21 by Saeed Saremi, Rupesh K. Srivastava, Saremi, Saeed +3 · 1 citation
Computer Science · #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning in Healthcare #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2303.11669

openalex publication_date 2023/03/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We consider the problem of generative modeling based on smoothing an unknown density of interest in ℝ^d using factorial kernels with M independent Gaussian channels with equal noise levels introduced by Saremi and Srivastava (2022). First, we fully characterize the time complexity of learning the resulting smoothed density in ℝ^Md, called M-density, by deriving a universal form for its parametrization in which the score function is by construction permutation equivariant. Next, we study the time complexity of sampling an M-density by analyzing its condition number for Gaussian distributions. This spectral analysis gives a geometric insight on the "shape" of M-densities as one increases M. Finally, we present results on the sample quality in this class of generative models on the CIFAR-10 dataset where we report Fréchet inception distances (14.15), notably obtained with a single noise level on long-run fast-mixing MCMC chains.

Cited by

• When Scores Learn Geometry: Rate Separations under the Manifold Hypothesis

Related

• Convergence of score-based generative modeling for general data distributions
• Generative Modeling with Denoising Auto-Encoders and Langevin Sampling
• Regularity of the score function in generative models
• A Good Score Does not Lead to A Good Generative Model
• Score-based generative models break the curse of dimensionality in learning a family of sub-Gaussian probability distributions