2012/05/31 by Luca Martino, Jesse Read, David Luengo · 101 citations
Computer Science · Mathematics · #Algorithm #Artificial intelligence #Bayesian Methods and Mixture Models #Bayesian probability #Computer science #Convergence (economics) #Gibbs sampling #Importance sampling #Machine learning #Markov Chains and Monte Carlo Methods #Markov chain #Markov chain Monte Carlo #Mathematical optimization #Mathematics #Metropolis–Hastings algorithm #Monte Carlo method #Piecewise #Sampling (signal processing) #Statistical Methods and Inference #Statistics #stat.CO #stat.ME
paper · pdf · doi:10.1109/tsp.2015.2420537
published in IEEE Transactions on Signal Processing 63(12), 3123-3138 (Institute of Electrical and Electronics Engineers) · Matlab code provided in http://a2rms.sourceforge.net/
arxiv created 2012/10/08 · openalex publication_date 2015/04/06 · arxiv updated 2015/11/20 · openalex created_date 2016/06/24 · openalex updated_date 2026/08/05
Bayesian methods have become very popular in signal processing lately, even though performing exact Bayesian inference is often unfeasible due to the lack of analytical expressions for optimal Bayesian estimators. In order to overcome this problem, Monte Carlo (MC) techniques are frequently used. Several classes of MC schemes have been developed, including Markov Chain Monte Carlo (MCMC) methods, particle filters and population Monte Carlo approaches. In this paper, we concentrate on the Gibbs-type approach, where automatic and fast samplers are needed to draw from univariate (full-conditional) densities. The Adaptive Rejection Metropolis Sampling (ARMS) technique is widely used within Gibbs sampling, but suffers from an important drawback: an incomplete adaptation of the proposal in some cases. In this work, we propose an alternative adaptive MCMC algorithm (IA2RMS) that overcomes this limitation, speeding up the convergence of the chain to the target, allowing us to simplify the construction of the sequence of proposals, and thus reducing the computational cost of the entire algorithm. Note that, although IA2RMS has been developed as an extremely efficient MCMC-within-Gibbs sampler, it also provides an excellent performance as a stand-alone algorithm when sampling from univariate distributions. In this case, the convergence of the proposal to the target is proved and a bound on the complexity of the proposal is provided. Numerical results, both for univariate (stand-alone IA2RMS) and multivariate (IA2RMS-within-Gibbs) distributions, show that IA2RMS outperforms ARMS and other classical techniques, providing a correlation among samples close to zero.