Rao-Blackwellized Score Matching on Manifolds

2026/05/31 by Divit Rawal
Mathematics · Computer Science · #stat.ML #cs.LG

paper · pdf

24 pages, 3 figures; SPIGM @ ICML 2026

arxiv created 2026/07/30 · arxiv updated 2026/07/31

Abstract

We study denoising score matching (DSM) when data are drawn from an embedded manifold M ⊂ ℝ^D. We show that under ambient Gaussian corruption, the target has variance that diverges as the noise scale decreases and correct for it by regressing against the conditional expectation given the nearest point projection on the manifold: the L²-optimal Rao-Blackwellized target. We then compute the small-noise expansion of this target and show that it recovers the true intrinsic Riemannian score to first order, with a second-order bias from a Tweedie term and two geometric terms dependent on how the manifold is embedded in ambient space: a curvature operator acting on the intrinsic score, and an additive drift generated by the spatial variation of the embedding's second fundamental form. On hyperspheres, we derive a simplified formula and show that both geometric terms vanish exactly on S², offering a theoretical explanation for why ambient DSM performs comparably to intrinsic methods on real Earth science spherical data in prior work.

Citations

• Generalization error bound for denoising score matching under relaxed manifold assumption
• Diffusion Models are Minimax Optimal Distribution Estimators
• Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions
• Convergence of denoising diffusion models under the manifold hypothesis
• Score-Based Generative Modeling through Stochastic Differential Equations
• Denoising Diffusion Probabilistic Models
• Generative Modeling by Estimating Gradients of the Data Distribution
• Tweedie’s Formula and Selection Bias
• A Connection Between Score Matching and Denoising Autoencoders

Related

• Generalization error bound for denoising score matching under relaxed manifold assumption
• Score Matching for Truncated Density Estimation on a Manifold
• Landing with the Score: Riemannian Optimization through Denoising
• A Connection Between Score Matching and Local Intrinsic Dimension
• When Scores Learn Geometry: Rate Separations under the Manifold Hypothesis