A Variational Perspective on Diffusion-Based Generative Models and Score Matching

2021/06/05 by Chin-Wei Huang, Huang, Chin-Wei, Jae Hyun Lim +3 · 31 citations
Computer Science · Physics and Astronomy · #Generative Adversarial Networks and Image Synthesis #Machine Learning in Healthcare #Model Reduction and Neural Networks #cs.LG

paper · pdf · doi:10.48550/arxiv.2106.02808

arxiv created 2021/09/30 · arxiv updated 2021/10/01

Abstract

Discrete-time diffusion-based generative models and score matching methods have shown promising results in modeling high-dimensional image data. Recently, Song et al. (2021) show that diffusion processes that transform data into noise can be reversed via learning the score function, i.e. the gradient of the log-density of the perturbed data. They propose to plug the learned score function into an inverse formula to define a generative diffusion process. Despite the empirical success, a theoretical underpinning of this procedure is still lacking. In this work, we approach the (continuous-time) generative diffusion directly and derive a variational framework for likelihood estimation, which includes continuous-time normalizing flows as a special case, and can be seen as an infinitely deep variational autoencoder. Under this framework, we show that minimizing the score-matching loss is equivalent to maximizing a lower bound of the likelihood of the plug-in reverse SDE proposed by Song et al. (2021), bridging the theoretical gap.

Citations

• Gradient-based learning applied to document recognition
• Auto-Encoding Variational Bayes

Cited by

• Score-based Generative Modeling in Latent Space
• Diffusion Models in Vision: A Survey
• LO-SDA: Latent Optimization for Score-based Atmospheric Data Assimilation
• An End-to-End Generative Diffusion Model for Heavy-Ion Collisions
• Ctrl-VI: Controllable Video Synthesis via Variational Inference
• Thermodynamic Performance Limits for Score-Based Diffusion Models
• Exact Solutions to the Quantum Schrödinger Bridge Problem
• A Mutil-conditional Diffusion Transformer for Versatile Seismic Wave Generation
• Autoregressive Diffusion Models
• DiffusionNFT: Online Diffusion Reinforcement with Forward Process
• Kuramoto Orientation Diffusion Models
• Simulating Diffusion Bridges with Score Matching
• Variational Diffusion Models
• Neural Bridge Processes
• Convergence of Deterministic and Stochastic Diffusion-Model Samplers: A Simple Analysis in Wasserstein Distance
• Diffusion models for inverse problems
• Concept-TRAK: Understanding how diffusion models learn concepts through concept-level attribution
• Style Transfer: A Decade Survey
• Inverse Design of Metamaterials with Manufacturing-Guiding Spectrum-to-Structure Conditional Diffusion Model
• RNE: plug-and-play diffusion inference-time control and energy-based training
• Denoising Diffusion Gamma Models
• SegDiff: Image Segmentation with Diffusion Probabilistic Models
• StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models
• Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis
• Score Matching With Missing Data
• Graph Flow Matching: Enhancing Image Generation with Neighbor-Aware Flow Fields
• Autoregressive regularized score-based diffusion models for multi-scenarios fluid flow prediction
• Combining Deterministic Enhanced Conditions with Dual-Streaming Encoding for Diffusion-Based Speech Enhancement
• Measurement Score-Based Diffusion Model
• Generalized Score Matching: Bridging f-Divergence and Statistical Estimation Under Correlated Noise
• A Complete Decomposition of Stochastic Differential Equations

Related

• Understanding Diffusion Models: A Unified Perspective
• Variational Diffusion Models
• Discrete vs. Continuous Trade-offs for Generative Models
• How Much is Enough? A Study on Diffusion Times in Score-based Generative Models
• Adaptivity and Convergence of Probability Flow ODEs in Diffusion Generative Models