General state space Markov chains and MCMC algorithms

2004/01/01 by Gareth O. Roberts, Jeffrey S. Rosenthal · 6 citations
Mathematics · #Algorithm #Applied mathematics #Computer science #Convergence (economics) #Ergodicity #Geometry #Markov Chains and Monte Carlo Methods #Markov chain #Markov chain Monte Carlo #Markov kernel #Markov model #Mathematical optimization #Mathematical proof #Mathematics #Monte Carlo method #State space #Statistical Methods and Inference #Statistical physics #Statistics #Stochastic processes and statistical mechanics #Variable-order Markov model #Weak convergence #math.PR

paper · pdf · doi:10.1214/154957804100000024

published as Probability Surveys 2004, Vol. 1, 20-71 · Published at http://dx.doi.org/10.1214/154957804100000024 in the Probability Surveys (http://www.i-journals.org/ps/) by the Institute of Mathematical Statistics (http://www.imstat.org)

openalex publication_date 2004/01/01 · arxiv created 2007/04/11 · arxiv updated 2009/12/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

This paper surveys various results about Markov chains on general (non-countable) state spaces. It begins with an introduction to Markov chain Monte Carlo (MCMC) algorithms, which provide the motivation and context for the theory which follows. Then, sufficient conditions for geometric and uniform ergodicity are presented, along with quantitative bounds on the rate of convergence to stationarity. Many of these results are proved using direct coupling constructions based on minorisation and drift conditions. Necessary and sufficient conditions for Central Limit Theorems (CLTs) are also presented, in some cases proved via the Poisson Equation or direct regeneration constructions. Finally, optimal scaling and weak convergence results for Metropolis-Hastings algorithms are discussed. None of the results presented is new, though many of the proofs are. We also describe some Open Problems.

Citations

• Improving Asymptotic Variance of MCMC Estimators: Non-reversible Chains are Better
• Quantitative bounds on convergence of time-inhomogeneous Markov chains
• Sufficient burn-in for Gibbs samplers for a hierarchical random effects model
• Exact sampling with coupled Markov chains and applications to statistical mechanics
• Equation of State Calculations by Fast Computing Machines
• Renewal theory and computable convergence rates for geometrically ergodic Markov chains
• Monte Carlo sampling methods using Markov chains and their applications
• Inference from Iterative Simulation Using Multiple Sequences
• Sampling-Based Approaches to Calculating Marginal Densities

Cited by

• Generalizations of Fano’s Inequality for Conditional Information Measures via Majorization Theory
• Software Framework for Tribotronic Systems
• Multilevel Delayed Acceptance MCMC with an Adaptive Error Model in PyMC3
• On micromodes in Bayesian posterior distributions and their implications for MCMC
• On the stability and ergodicity of adaptive scaling Metropolis algorithms
• On the occupancy problem for a regime-switching model