Sequential Monte Carlo approximations of Wasserstein--Fisher--Rao gradient flows

2025/06/06 by Francesca R. Crucinio, Crucinio, Francesca R., Sahani Pathiraja +1 · 1 voice · 2 citations
Computer Science · Mathematics · #Stochastic Gradient Optimization Techniques #Markov Chains and Monte Carlo Methods #Geometric Analysis and Curvature Flows

paper · pdf · doi:10.48550/arxiv.2506.05905

Abstract

We consider the problem of sampling from a probability distribution π. It is well known that this can be written as an optimisation problem over the space of probability distribution in which we aim to minimise the Kullback--Leibler divergence from π. We consider several partial differential equations (PDEs) whose solution is a minimiser of the Kullback--Leibler divergence from π and connect them to well-known Monte Carlo algorithms. We focus in particular on PDEs obtained by considering the Wasserstein--Fisher--Rao geometry over the space of probabilities and show that these lead to a natural implementation using importance sampling and sequential Monte Carlo. We propose a novel algorithm to approximate the Wasserstein--Fisher--Rao flow of the Kullback--Leibler divergence and conduct an extensive empirical study to identify when these algorithms outperforms other popular Monte Carlo algorithms.

Citations

• Evolution of Gaussians in the Hellinger-Kantorovich-Boltzmann gradient flow
• Connections between sequential Bayesian inference and evolutionary dynamics
• Provable Convergence and Limitations of Geometric Tempering for Langevin Dynamics
• NETS: A Non-Equilibrium Transport Sampler
• Stochastic mirror descent for nonparametric adaptive importance sampling
• Stein transport for Bayesian inference
• Optimised Annealed Sequential Monte Carlo Samplers
• Provable Benefit of Annealed Langevin Monte Carlo for Non-log-concave Sampling
• Fisher-Rao Gradient Flow: Geodesic Convexity and Functional Inequalities
• Efficient, Multimodal, and Derivative-Free Bayesian Inference With Fisher-Rao Gradient Flows
• Error bounds for particle gradient descent, and extensions of the log-Sobolev and Talagrand inequalities
• Measure transport with kernel mean embeddings
• Sampling in Unit Time with Kernel Fisher-Rao Flow
• Quantifying the effectiveness of linear preconditioning in Markov chain Monte Carlo
• A connection between Tempering and Entropic Mirror Descent
• Sampling via Gradient Flows in the Space of Probability Measures
• Properties of Marginal Sequential Monte Carlo Methods
• An Explicit Expansion of the Kullback-Leibler Divergence along its Fisher-Rao Gradient Flow
• Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance
• Learning Gaussian Mixtures Using the Wasserstein-Fisher-Rao Gradient Flow
• Mirror Descent with Relative Smoothness in Measure Spaces, with application to Sinkhorn and EM
• Variational inference via Wasserstein gradient flows
• Particle algorithms for maximum likelihood training of latent variable models
• Adaptive Importance Sampling meets Mirror Descent: a Bias-variance\n tradeoff
• An invitation to sequential Monte Carlo samplers
• Generalized Bayesian Filtering via Sequential Monte Carlo
• Collective Proposal Distributions for Nonlinear MCMC samplers: Mean-Field Theory and Fast Implementation
• High-Order Langevin Diffusion Yields an Accelerated MCMC Algorithm
• Accelerating Langevin Sampling with Birth-death
• Rapid Convergence of the Unadjusted Langevin Algorithm: Isoperimetry\n Suffices
• Large Sample Asymptotics of the Pseudo-Marginal Method
• Analysis of Langevin Monte Carlo via convex optimization
• Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problem
• A probabilistic interpretation of replicator-mutator dynamics
• Negative association, ordering and convergence of resampling methods
• Stein Variational Gradient Descent: A General Purpose Bayesian Inference\n Algorithm
• A JKO splitting scheme for Kantorovich-Fisher-Rao gradient flows
• Variational Inference: A Review for Statisticians
• Importance Sampling: Intrinsic Dimension and Computational Cost
• Provable Bayesian Inference via Particle Mirror Descent
• Theoretical guarantees for approximate sampling from smooth and log-concave densities
• Information Geometry and Evolutionary Game Theory
• The Replicator Equation as an Inference Dynamic
• Toward Practical N2 Monte Carlo: the Marginal Particle Filter
• Central limit theorem for sequential Monte Carlo methods and its application to Bayesian inference
• Sequential monte carlo samplers
• Optimal scaling for various Metropolis-Hastings algorithms
• Annealed Importance Sampling
• Annealed importance sampling

Cited by

• An operator splitting analysis of Wasserstein--Fisher--Rao gradient flows
• A note on the unique properties of the Kullback--Leibler divergence for sampling via gradient flows

Discussions

• Motivated by the reading of this nice article: "Sequential Monte Carlo approximations of Wasserstein--Fisher--Rao gradient flows" by Francesca R. Crucinio, Sahani Pathiraja arxiv.org/abs/2506.05905 [bsky, 9 points, 1 comments]

Related

• An Explicit Expansion of the Kullback-Leibler Divergence along its Fisher-Rao Gradient Flow
• A note on the unique properties of the Kullback--Leibler divergence for sampling via gradient flows
• Sampling via Gradient Flows in the Space of Probability Measures
• Gradient Flows for Sampling: Mean-Field Models, Gaussian Approximations and Affine Invariance
• Gradient Flow Sampler-based Distributionally Robust Optimization