2019/08/30 by Samuel Livingstone, Giacomo Zanella, Livingstone, Samuel +1 · 8 citations
Computer Science · Mathematics · #Bayesian Methods and Mixture Models #Computation (stat.CO) #FOS: Computer and information sciences #Markov Chains and Monte Carlo Methods #Methodology (stat.ME) #Statistical Methods and Inference #stat.CO #stat.ME
paper · pdf · doi:10.48550/arxiv.1908.11812
openalex publication_date 2019/08/30 · arxiv created 2020/05/11 · arxiv updated 2020/05/12 · openalex created_date 2022/07/28 · openalex updated_date 2026/07/28
There is a tension between robustness and efficiency when designing Markov chain Monte Carlo (MCMC) sampling algorithms. Here we focus on robustness with respect to tuning parameters, showing that more sophisticated algorithms tend to be more sensitive to the choice of step-size parameter and less robust to heterogeneity of the distribution of interest. We characterise this phenomenon by studying the behaviour of spectral gaps as an increasingly poor step-size is chosen for the algorithm. Motivated by these considerations, we propose a novel and simple gradient-based MCMC algorithm, inspired by the classical Barker accept-reject rule, with improved robustness properties. Extensive theoretical results, dealing with robustness to tuning, geometric ergodicity and scaling with dimension, suggest that the novel scheme combines the robustness of simple schemes with the efficiency of gradient-based ones. We show numerically that this type of robustness is particularly beneficial in the context of adaptive MCMC, giving examples where our proposed scheme significantly outperforms state-of-the-art alternatives.