Stochastic Gradient MCMC Methods for Hidden Markov Models

2017/06/14 by Yi-An Ma, Ma, Yi-An, Nicholas J. Foti +3
Computer Science · Mathematics · #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (stat.ML) #Machine Learning and Algorithms #Markov Chains and Monte Carlo Methods

paper · pdf · doi:10.48550/arxiv.1706.04632

openalex publication_date 2017/06/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Stochastic gradient MCMC (SG-MCMC) algorithms have proven useful in scaling Bayesian inference to large datasets under an assumption of i.i.d data. We instead develop an SG-MCMC algorithm to learn the parameters of hidden Markov models (HMMs) for time-dependent data. There are two challenges to applying SG-MCMC in this setting: The latent discrete states, and needing to break dependencies when considering minibatches. We consider a marginal likelihood representation of the HMM and propose an algorithm that harnesses the inherent memory decay of the process. We demonstrate the effectiveness of our algorithm on synthetic experiments and an ion channel recording data, with runtimes significantly outperforming batch MCMC.

Citations

• Random dynamical systems
• A Stochastic Approximation Method
• A tutorial on hidden Markov models and selected applications in speech recognition

Related

• Stochastic Gradient MCMC for State Space Models
• Stochastic Variational Inference for Hidden Markov Models
• Efficient and Generalizable Tuning Strategies for Stochastic Gradient MCMC
• Stochastic Gradient MCMC for Nonlinear State Space Models
• Stochastic gradient Markov chain Monte Carlo