New affine invariant ensemble samplers and their dimensional scaling

2025/05/05 by Yifan Chen, Chen, Yifan · 3 citations
Computer Science · Mathematics · Agricultural and Biological Sciences · #Bayesian Methods and Mixture Models #Advanced Statistical Methods and Models #Advanced Scientific Research Methods

paper · pdf · doi:10.48550/arxiv.2505.02987

Abstract

We introduce new affine invariant ensemble Markov chain Monte Carlo (MCMC) samplers that are easy to construct and improve upon existing methods, especially for high-dimensional problems. We first propose a simple derivative-free side move sampler that improves upon popular samplers in the emcee package by generating more effective proposal directions. We then develop a class of derivative-based affine invariant ensemble Hamiltonian Monte Carlo (HMC) samplers based on antisymmetric preconditioning using complementary ensembles, which outperform standard, non-affine-invariant HMC when sampling highly anisotropic distributions. We provide asymptotic scaling analysis for high-dimensional Gaussian targets to further elucidate the properties of these affine invariant ensemble samplers. In particular, with derivative information, the affine invariant ensemble HMC can scale much better with dimension compared to derivative-free ensemble samplers.

Citations

• The No-Underrun Sampler: A Locally-Adaptive, Gradient-Free MCMC Method
• Stable Derivative Free Gaussian Mixture Variational Inference for Bayesian Inverse Problems
• Running Markov Chain Monte Carlo on Modern Hardware and Software
• Efficient, Multimodal, and Derivative-Free Bayesian Inference With Fisher-Rao Gradient Flows
• Ensemble-Based Annealed Importance Sampling
• Sampling via Gradient Flows in the Space of Probability Measures
• Hamiltonian Monte Carlo for efficient Gaussian sampling: long and random steps
• Second Order Ensemble Langevin Method for Sampling and Inverse Problems
• Efficient Derivative-free Bayesian Inference for Large-Scale Inverse Problems
• A gradient-free subspace-adjusting ensemble sampler for infinite-dimensional Bayesian inverse problems
• Sampling with Riemannian Hamiltonian Monte Carlo in a Constrained Space
• Focusing on Difficult Directions for Learning HMC Trajectory Lengths
• Ensemble Markov chain Monte Carlo with teleporting walkers
• Exponential ergodicity of mirror-Langevin diffusions
• Affine invariant interacting Langevin dynamics for Bayesian inference
• Fokker-Planck particle systems for Bayesian inference: Computational approaches
• Wasserstein stability estimates for covariance-preconditioned Fokker-Planck equations
• Accelerating Langevin Sampling with Birth-death
• Optimal Scaling of Random-Walk Metropolis Algorithms on General Target Distributions
• Convergence Rate of Riemannian Hamiltonian Monte Carlo and Faster\n Polytope Volume Computation
• Ensemble preconditioning for Markov chain Monte Carlo simulation
• Stochastic Quasi-Newton Langevin Monte Carlo
• Properties of the Affine Invariant Ensemble Sampler in high dimensions
• emcee : The MCMC Hammer
• The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo
• Optimal tuning of the Hybrid Monte-Carlo Algorithm
• A Markov Chain Monte Carlo version of the genetic algorithm Differential Evolution: easy Bayesian computing for real parameter spaces
• Weak convergence and optimal scaling of random walk Metropolis algorithms
• A Simplex Method for Function Minimization

Cited by

• Scale-Adaptive Generative Flows for Multiscale Scientific Data
• Lipschitz-Guided Design of Interpolation Schedules in Generative Models
• The Within-Orbit Adaptive Leapfrog No-U-Turn Sampler

Related

• Properties of the Affine Invariant Ensemble Sampler in high dimensions
• A gradient-free subspace-adjusting ensemble sampler for infinite-dimensional Bayesian inverse problems
• ATLAS: Adapting Trajectory Lengths and Step-Size for Hamiltonian Monte Carlo
• emcee : The MCMC Hammer
• Hamiltonian Monte Carlo for efficient Gaussian sampling: long and random steps