Optimised Annealed Sequential Monte Carlo Samplers

2024/08/22 by Saifuddin Syed, Syed, Saifuddin, Alexandre Bouchard‐Côté +5 · 17 citations
Decision Sciences · Mathematics · #Computation (stat.CO) #FOS: Computer and information sciences #Simulation Techniques and Applications #Statistical Methods and Inference

paper · pdf · doi:10.48550/arxiv.2408.12057

openalex publication_date 2024/08/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Annealed Sequential Monte Carlo (ASMC) samplers are special cases of SMC samplers where the sequence of distributions can be embedded in a smooth path of distributions. Using this underlying path and a performance model based on the variance of the normalising constant estimator, we systematically study dense-schedule limits. From our theory emerges a notion of global barrier, capturing the inherent complexity of normalising constant approximation under our performance model. We then turn the resulting approximations into surrogate objective functions of algorithm performance, using them to guide method development. This leads to novel adaptive methods, Optimised Annealed SMC (OASMC), which address practical difficulties inherent in previous adaptive SMC methods. First, our OASMC algorithms are predictable: they produce a sequence of increasingly precise estimates at deterministic, known times. Second, Optimised Annealed Importance Sampling (OAIS), a special case of OASMC, enables schedule adaptation at a memory cost constant in the number of particles, requiring significantly less communication. Finally, these characteristics make OAIS highly efficient on GPUs. We provide an open-source, high-performance GPU implementation of our method and demonstrate up to a hundred-fold speed improvement compared to state-of-the-art adaptive AIS methods.

Cited by

• Test-time scaling of diffusions with flow maps
• Convergence of a Sequential Monte Carlo algorithm towards multimodal distributions on Rd
• An operator splitting analysis of Wasserstein--Fisher--Rao gradient flows
• Exploring Pareto smoothing in sequential Monte Carlo
• Accelerated Sequential Posterior Inference via Reuse for Gravitational-Wave Analyses
• Machine Learning the Conformal Manifold of Holographic CFT₂s
• Learning Boltzmann Generators via Constrained Mass Transport
• Reinforced sequential Monte Carlo for amortised sampling
• CREPE: Controlling Diffusion with Replica Exchange
• Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inference
• The Cosine Schedule is Fisher-Rao-Optimal for Masked Discrete Diffusion Models
• Polynomial complexity sampling from multimodal distributions using Sequential Monte Carlo
• Sequential Monte Carlo approximations of Wasserstein--Fisher--Rao gradient flows
• Asymptotically exact variational flows via involutive MCMC kernels
• Restricted Spectral Gap Decomposition for Simulated Tempering Targeting Mixture Distributions
• Scalable Bayesian Monte Carlo: fast uncertainty estimation beyond deep ensembles
• Sampling through iterated approximation: Gradient-free and multi-fidelity Bayesian inference via transport

Related

• Posterior Exploration based Sequential Monte Carlo for Global Optimization
• Toward Unlimited Self-Learning MCMC with Parallel Adaptive Annealing
• Incorporating the ChEES Criterion into Sequential Monte Carlo Samplers
• Adaptive Annealed Importance Sampling with Constant Rate Progress
• Tuning Sequential Monte Carlo Samplers via Greedy Incremental Divergence Minimization