Consensus Based Sampling

2021/06/01 by José A. Carrillo, J. A. Carrillo, Carrillo, J. A. +9 · 14 citations
Computer Science · Mathematics · #Affine transformation #Algorithm #Applied mathematics #Bayesian Methods and Mixture Models #Computer science #Estimator #Euclidean space #Gaussian #Gaussian Processes and Bayesian Inference #Importance sampling #Laplace transform #Laplace's method #Markov Chains and Monte Carlo Methods #Mathematical analysis #Mathematical optimization #Mathematics #Monte Carlo method #Parametric statistics #Sampling (signal processing) #cs.NA #math.DS #math.NA #msc:35G25 #msc:62F15 #msc:65C35 #msc:65N21

paper · pdf · open access · doi:10.1111/sapm.12470

published in Studies in Applied Mathematics 148(3), 1069-1140 (Wiley)

arxiv created 2021/11/04 · arxiv updated 2021/11/05 · openalex publication_date 2022/01/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

We propose a novel method for sampling and optimization tasks based on a stochastic interacting particle system. We explain how this method can be used for the following two goals: (i) generating approximate samples from a given target distribution; (ii) optimizing a given objective function. The approach is derivative-free and affine invariant, and is therefore well-suited for solving inverse problems defined by complex forward models: (i) allows generation of samples from the Bayesian posterior and (ii) allows determination of the maximum a posteriori estimator. We investigate the properties of the proposed family of methods in terms of various parameter choices, both analytically and by means of numerical simulations. The analysis and numerical simulation establish that the method has potential for general purpose optimization tasks over Euclidean space; contraction properties of the algorithm are established under suitable conditions, and computational experiments demonstrate wide basins of attraction for various specific problems. The analysis and experiments also demonstrate the potential for the sampling methodology in regimes in which the target distribution is unimodal and close to Gaussian; indeed we prove that the method recovers a Laplace approximation to the measure in certain parametric regimes and provide numerical evidence that this Laplace approximation attracts a large set of initial conditions in a number of examples.

Citations

• Monte Carlo sampling methods using Markov chains and their applications

Cited by

• Treasure Search Optimization
• Mean-Field Model for Two-Layer Neural Networks Trained with Consensus-Based Optimization
• Blade: A Derivative-free Bayesian Inversion Method using Diffusion Priors
• Consensus-Based Optimization Beyond Finite-Time Analysis
• Random dynamical systems for McKean--Vlasov SDEs via rough path theory
• Consensus-based optimization for closed-box adversarial attacks and a connection to evolution strategies
• Affine invariant interacting Langevin dynamics in Markov chain importance sampling for rare event estimation
• Parallel computations for Metropolis Markov chains with Picard maps
• A localized consensus-based sampling algorithm
• Uniform-in-time propagation of chaos for Consensus-Based Optimization
• A new gradient-free active subspace estimation method with application to rare event probability estimation
• Gradient-Free Sequential Bayesian Experimental Design via Interacting Particle Systems
• Mathematical Analysis of the PDE Model for the Consensus-based Optimization
• Controlled stochastic processes for simulated annealing

Related

• A localized consensus-based sampling algorithm
• Metropolis-adjusted interacting particle sampling
• Bayesian sampling using interacting particles
• Consensus-based algorithms for stochastic optimization problems
• Learning Rate Free Sampling in Constrained Domains