2020/03/02 by Junfeng Wen, Bo Dai, Wen, Junfeng +5 · 1 citation
Business, Management and Accounting · Decision Sciences · Mathematics · #Advanced Queuing Theory Analysis #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Markov Chains and Monte Carlo Methods #Simulation Techniques and Applications
paper · pdf · doi:10.48550/arxiv.2003.00722
openalex publication_date 2020/03/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We consider the problem of approximating the stationary distribution of an ergodic Markov chain given a set of sampled transitions. Classical simulation-based approaches assume access to the underlying process so that trajectories of sufficient length can be gathered to approximate stationary sampling. Instead, we consider an alternative setting where a fixed set of transitions has been collected beforehand, by a separate, possibly unknown procedure. The goal is still to estimate properties of the stationary distribution, but without additional access to the underlying system. We propose a consistent estimator that is based on recovering a correction ratio function over the given data. In particular, we develop a variational power method (VPM) that provides provably consistent estimates under general conditions. In addition to unifying a number of existing approaches from different subfields, we also find that VPM yields significantly better estimates across a range of problems, including queueing, stochastic differential equations, post-processing MCMC, and off-policy evaluation.