Likelihood Ratio Gradient Estimation for Steady-State Parameters

2017/07/09 by Peter W. Glynn, Glynn, Peter W., Mariana Olvera-Cravioto +1 · 1 citation
Mathematics · #60G42 #60J22 #FOS: Mathematics #Primary: 65C05 #Probability (math.PR) #Secondary: 60J10 #math.PR #msc:60G42 #msc:60J10 #msc:60J22 #msc:65C05

paper · pdf · doi:10.48550/arxiv.1707.02659

arxiv created 2018/03/09 · arxiv updated 2018/03/12

Abstract

We consider a discrete-time Markov chain \boldsymbolΦ on a general state-space \sf X, whose transition probabilities are parameterized by a real-valued vector \boldsymbolθ. Under the assumption that \boldsymbolΦ is geometrically ergodic with corresponding stationary distribution π(\boldsymbolθ), we are interested in estimating the gradient ∇ α(\boldsymbolθ) of the steady-state expectation α(\boldsymbolθ) = π( \boldsymbolθ) f. To this end, we first give sufficient conditions for the differentiability of α(\boldsymbolθ) and for the calculation of its gradient via a sequence of finite horizon expectations. We then propose two different likelihood ratio estimators and analyze their limiting behavior.

Cited by

• Source identification via pathwise gradient estimation

Related

• Robust estimation for ergodic Markovian processes
• Convergence of the likelihood ratio method for linear response of non-equilibrium stationary states
• Maximum likelihood estimation for nonembeddable Markov chains when the cycle length is shorter than the data observation interval
• Correct Approximation of Stationary Distributions
• Steady State Sensitivity Analysis of Continuous Time Markov Chains