Understanding the Metropolis-Hastings Algorithm

1995/11/01 by Siddhartha Chib, Edward Greenberg · 3,730 citations
Computer Science · Mathematics · #Algorithm #Artificial intelligence #Bayesian Methods and Mixture Models #Bayesian probability #Computer science #Gibbs sampling #Hybrid Monte Carlo #Machine learning #Markov Chains and Monte Carlo Methods #Markov chain #Markov chain Monte Carlo #Mathematics #Metropolis–Hastings algorithm #Rejection sampling #Sampling (signal processing) #Statistical Methods and Bayesian Inference

paper · doi:10.1080/00031305.1995.10476177

published in The American Statistician 49(4), 327-335 (Taylor & Francis)

openalex publication_date 1995/11/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/31

Abstract

We provide a detailed, introductory exposition of the Metropolis-Hastings algorithm, a powerful Markov chain method to simulate multivariate distributions. A simple, intuitive derivation of this method is given along with guidance on implementation. Also discussed are two applications of the algorithm, one for implementing acceptance-rejection sampling when a blanketing function is not available and the other for implementing the algorithm with block-at-a-time scans. In the latter situation, many different algorithms, including the Gibbs sampler, are shown to be special cases of the Metropolis-Hastings algorithm. The methods are illustrated with examples.

Cited by

• Markov Chain Monte Carlo in Practice: A Roundtable Discussion
• A History of the Metropolis–Hastings Algorithm
• Bayesian Design of “Successful” Replications
• The Long-Term Effect of Marketing Strategy on Brand Sales
• Higher-Order Latent Trait Models for Cognitive Diagnosis
• LBA: Textual Hard-Label Adversarial Attack under Low Query Budgets
• A mixture model for responses and response times with a higher‐order ability structure to detect rapid guessing behaviour
• Parallel hierarchical sampling: a practical multiple-chains sampler for Bayesian model selection
• The Probabilistic Analysis of the Communication Network created by Dynamic Boundary Coverage
• Medical ultrasound full waveform inversion via Stein variational inference: posterior mean reconstruction and uncertainty quantification
• MCMC Guided CNN Training and Segmentation for Pancreas Extraction
• A Note on Using Discretized Simulated Data to Estimate Implicit Likelihoods in Bayesian Analyses
• Robust Analytics for Video-Based Gait Biometrics
• A Critical Review of Monte Carlo Algorithms Balancing Performance and Probabilistic Accuracy with AI Augmented Framework
• Variational Inference for Fully Bayesian Hierarchical Linear Models
• Alternatives for Generating a Reduced Basis to Solve the Hyperspectral Diffuse Optical Tomography Model
• Row-Stochastic Matrices Can Provably Outperform Doubly Stochastic Matrices in Decentralized Learning
• Synthetic Curriculum Reinforces Compositional Text-to-Image Generation
• Luminance-Aware Statistical Quantization: Unsupervised Hierarchical Learning for Illumination Enhancement
• Autocovariance and Optimal Design for Random Walk Metropolis-Hastings Algorithm
• Revealing the Hidden Third Dimension of Point Defects in Two-Dimensional MXenes
• Conditioning on posterior samples for flexible frequentist goodness-of-fit testing
• Spectral-Convergent Decentralized Machine Learning: Theory and Application in Space Networks
• ESA: Energy-Based Shot Assembly Optimization for Automatic Video Editing
• Moderating Factors of Immediate, Gross, and Net Cross-Brand Effects of Price Promotions
• Maximin design on non hypercube domain and kernel interpolation
• Statistical Inversion Using Sparsity and Total Variation Prior And Monte Carlo Sampling Method For Diffuse Optical Tomography
• Half-wave Plates for the Spider Cosmic Microwave Background Polarimeter
• A Bayesian Nonparametric Meta-Analysis Model
• A tutorial on Bayesian models of perception
• A Representative Sampling Method for Peer Encouragement Designs in Network Experiments
• Static Analysis for Probabilistic Programs
• SCORENF: Score-based Normalizing Flows for Sampling Unnormalized distributions
• Non-stabilizerness as a Diagnostic of Criticality and Exceptional Points in Non-Hermitian Spin Chains
• Table-Top Scene Analysis Using Knowledge-Supervised MCMC
• Isolating Compiler Bugs through Compilation Steps Analysis
• Bayesian inference of spectral induced polarization parameters for laboratory complex resistivity measurements of rocks and soils
• TGPO: Temporal Grounded Policy Optimization for Signal Temporal Logic Tasks
• Spectral gap of Metropolis-within-Gibbs under log-concavity
• Synthetic Dialogue Generation for Interactive Conversational Elicitation & Recommendation (ICER)
• Bayesian Gamma-Negative Binomial Modeling of Single-Cell RNA Sequencing Data
• Plastic deformation as a phase transition: A combinatorial model of plastic flow in copper single crystals
• A new algorithm for extracting a small representative subgraph from a very large graph
• Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces
• Modeling Missing Response Data in Item Response Theory: Addressing Missing Not at Random Mechanism with Monotone Missing Characteristics
• Sequential Design for the Efficient Estimation of Offshore Structure Failure Probability
• Rare Event Simulation of Quantum Error-Correcting Circuits
• MC²RAM: Markov Chain Monte Carlo Sampling in SRAM for Fast Bayesian Inference
• From Federated Learning to X-Learning: Breaking the Barriers of Decentrality Through Random Walks
• An unsupervised bayesian approach for the joint reconstruction and classification of cutaneous reflectance confocal microscopy images
• Sampling-based Bayesian Inference with gradient uncertainty
• Gaussian process surrogate with physical law-corrected prior for multi-coupled PDEs defined on irregular geometry
• Multiple target tracking with interaction using an MCMC MRF Particle Filter
• Energy-Efficient Wireless LLM Inference via Uncertainty and Importance-Aware Speculative Decoding
• Bayesian sparse graphical models and their mixtures using lasso selection priors
• Achieving High Coverage for Floating-point Code via Unconstrained Programming (Extended Version)
• Bayesian variable selection for spatially dependent generalized linear models
• Introduction to Relational Networks for Classification
• The Virtual Patch Clamp: Imputing C. elegans Membrane Potentials from Calcium Imaging
• SkipTrack: A Bayesian Hierarchical Model for Self-tracked Menstrual Cycle Length and Regularity in Large Mobile Health Cohorts
• Geographically-dependent individual-level models for infectious diseases transmission
• Bayesian Neural Network Surrogates for Bayesian Optimization of Carbon Capture and Storage Operations
• Bayesian averaging of computer models with domain discrepancies: a nuclear physics perspective
• Quantifying Program Bias
• On the Unicity of Smartphone Applications
• Generalizing Informed Sampling for Asymptotically Optimal Sampling-based Kinodynamic Planning via Markov Chain Monte Carlo
• Spatial Clustering Regression of Count Value Data via Bayesian Mixture of Finite Mixtures
• Varying the resolution of the Rouse model on temporal and spatial scales: application to multiscale modelling of DNA dynamics
• Markov-Chain Monte-Carlo A Bayesian Approach to Statistical Mechanics
• Bayesian analysis of multivariate stable distributions using one-dimensional projections
• Bayesian characterization of uncertainties surrounding fluvial flood hazard estimates
• Modelling Dependent Fates of Long-Term Mates in Mark-Recapture Studies Using Conditional Data Methods
• Bayesian DEJD model and detection of asymmetric jumps
• Point and Interval Estimation of Powers of Scale Parameters for Two Normal Populations with a Common Mean
• Maximum Likelihood Estimation for Single Linkage Hierarchical Clustering
• InferSpark: Statistical Inference at Scale
• Learning to Control Dynamical Agents via Spiking Neural Networks and Metropolis-Hastings Sampling
• Fast Gaussian Processes under Monotonicity Constraints
• Warranty Cost Estimation Using Bayesian Network
• Interactive Multi-Objective Probabilistic Preference Learning with Soft and Hard Bounds
• Motor Insurance Accidental Damage Claims Modeling with Factor Collapsing and Bayesian Model Averaging
• Tractable Post-Selection Maximum Likelihood Inference for the Lasso
• Schrödinger-Föllmer Sampler: Sampling without Ergodicity
• Style Imitation and Chord Invention in Polyphonic Music with Exponential Families
• Social physics
• Metropolis-Hastings Sampling for 3D Gaussian Reconstruction
• A Tutorial on Reversible Jump MCMC with a View toward Applications in QTL‐mapping
• Course Project Report: Comparing MCMC and Variational Inference for Bayesian Probabilistic Matrix Factorization on the MovieLens Dataset
• Spatial Weibull Regression with Multivariate Log Gamma Process and Its Applications to China Earthquake Economic Loss
• "Don't Do That!": Guiding Embodied Systems through Large Language Model-based Constraint Generation
• Bayesian full waveform inversion with sequential surrogate model refinement
• Simultaneous estimation of complementary moment independent sensitivity measures for reliability analysis
• A Repelling-Attracting Metropolis Algorithm for Multimodality
• Logical Abstractions for Noisy Variational Quantum Algorithm Simulation
• Synthesizing Queries via Interactive Sketching
• dentist: Quantifying uncertainty by sampling points around maximum likelihood estimates
• Bayesian inference on the isotopic building blocks of Mars and Earth
• Service Chain Composition with Failures in NFV Systems: A Game-Theoretic Perspective
• Combining Bayesian Inference and Reinforcement Learning for Agent Decision Making: A Review
• Nonparametric Density Estimation under Adversarial Losses
• Estimating Extreme Drought Risk Through Classical and Bayesian Paradigms
• Analyzing MCMC Output
• Extracting Falsifiable Predictions from Sloppy Models
• Analysis of high dimensional multivariate stochastic volatility models
• Analysis of treatment response data without the joint distribution of potential outcomes
• Additive cubic spline regression with Dirichlet process mixture errors
• Bayesian Sparse Propensity Score Estimation for Unit Nonresponse
• Classification of Computer Models with Labelled Outputs
• Reliable entanglement verification
• Prevalence of positive selection among nearly neutral amino acid replacements in Drosophila
• Economic and agronomic impact of commercialized GM crops: a meta-analysis
• Chapter 15 Monte carlo simulation and numerical integration
• Nested Sampling with Slice-within-Gibbs: Efficient Evidence Calculation for Hierarchical Bayesian Models
• A Multi-Language Perspective on the Robustness of LLM Code Generation
• Bayesian Hierarchical Modeling: Application Towards Production Results in the Eagle Ford Shale of South Texas
• Analysis of multivariate probit models
• Bayesian Nonlinear Models for Repeated Measurement Data: An Overview, Implementation, and Applications
• Estimation of COVID-19 spread curves integrating global data and borrowing information
• Cubing for Tuning
• Estimation via Markov chain Monte Carlo
• Auto-Regressive Standard Precipitation Index: A Bayesian Approach for Drought Characterization
• The Impact of Move Schemes on Simulated Annealing Performance
• Results from a Search for Dark Matter in the Complete LUX Exposure
• A general Bayesian framework for foreground modelling and chromaticity correction for global 21 cm experiments
• Bayesian Inference of Quasi‐Linear Radial Diffusion Parameters using Van Allen Probes
• Speedup Chip Yield Analysis by Improved Quantum Bayesian Inference
• Bayesian Inference for Transportation Origin–Destination Matrices: the Poisson–Inverse Gaussian and Other Poisson Mixtures
• Particle-based fluid dynamics: Comparison of different Bhatnagar-Gross-Krook models and the direct simulation Monte Carlo method for hypersonic flows
• Bayesian inference and non-linear extensions of the CIRCE method for quantifying the uncertainty of closure relationships integrated into thermal-hydraulic system codes
• Mapping the kinetic Sunyaev-Zel’dovich effect toward MACS J0717.5+3745 with NIKA
• Mobile Collaborative Spectrum Sensing for Heterogeneous Networks: A Bayesian Machine Learning Approach
• Mathematical Analysis of the PDE Model for the Consensus-based Optimization
• Bayesian Analysis of Interpretable Aging across Thousands of Lithium-ion Battery Cycles
• The exchangeable multinomial model as an approach to testing deterministic axioms of choice and measurement
• Safe Interactions via Monte Carlo Linear-Quadratic Games
• Effects of Surface Corrugation on Gas-Surface Scattering and Macroscopic Flows
• A Bayesian approach for instrumental variable analysis with censored time-to-event outcome. [europepmc]
• Efficient Characterization of Parametric Uncertainty of Complex (Bio)chemical Networks. [europepmc]
• Parameterizing Spatial Models of Infectious Disease Transmission that Incorporate Infection Time Uncertainty Using Sampling-Based Likelihood Approximations. [europepmc]
• AxonPacking: An Open-Source Software to Simulate Arrangements of Axons in White Matter. [europepmc]
• Statistical emulation of landslide-induced tsunamis at the Rockall Bank, NE Atlantic. [europepmc]
• A Bayesian approach to modelling the impact of hydrodynamic shear stress on biofilm deformation. [europepmc]
• Non-significant p-values? Strategies to understand and better determine the importance of effects and interactions in logistic regression. [europepmc]
• Scalable nonlinear programming framework for parameter estimation in dynamic biological system models. [europepmc]
• Multivariate Higher-Order IRT Model and MCMC Algorithm for Linking Individual Participant Data From Multiple Studies. [europepmc]
• Integrating transcriptomic network reconstruction and eQTL analyses reveals mechanistic connections between genomic architecture and Brassica rapa development. [europepmc]
• Parallel Tempering with Lasso for model reduction in systems biology. [europepmc]
• Bayesian Estimation of the DINA Model With Pólya-Gamma Gibbs Sampling. [europepmc]
• An Intelligence EEG Signal Recognition Method via Noise Insensitive TSK Fuzzy System Based on Interclass Competitive Learning. [europepmc]
• Community stakeholder preferences for evidence-based practice implementation strategies in behavioral health: a best-worst scaling choice experiment. [europepmc]
• Methods to Develop an in silico Clinical Trial: Computational Head-to-Head Comparison of Lisdexamfetamine and Methylphenidate. [europepmc]
• Induced fit with replica exchange improves protein complex structure prediction. [europepmc]
• Analyzing differences between parent- and self-report measures with a latent space approach. [europepmc]
• Filter inference: A scalable nonlinear mixed effects inference approach for snapshot time series data. [europepmc]
• Modelling the spread and mitigation of an emerging vector-borne pathogen: Citrus greening in the U.S. [europepmc]
• Analysis and design of single-cell experiments to harvest fluctuation information while rejecting measurement noise. [europepmc]
• Uncertainty quantification in multivariable regression for material property prediction with Bayesian neural networks. [europepmc]
• Value construction through sequential sampling explains serial dependencies in decision making. [europepmc]
• Fast analysis of biobank-size data and meta-analysis using the BGLR R-package. [europepmc]
• Generative artificial intelligence: a historical perspective. [europepmc]
• Suppression of COVID-19 outbreak in the municipality of Vo’, Italy [europepmc]

Related

• Regression Quantiles
• The Nonparametric Approach to Demand Analysis
• Experts: Which Ones Should You Trust?
• Risk, Return, and Equilibrium: Empirical Tests
• Resource-Constrained versus Demand-Constrained Systems