A Short Proof of a Convex Representation for Stationary Distributions of Markov Chains with an Application to State Space Truncation

2022/08/06 by Zeyu Zheng, Zheng, Zeyu, Alex Infanger +3
Mathematics · Computer Science · #Markov Chains and Monte Carlo Methods #Random Matrices and Applications #Bayesian Modeling and Causal Inference

paper · pdf · doi:10.48550/arxiv.2208.03446

Abstract

In an influential paper, Courtois and Semal (1984) establish that when G is an irreducible substochastic matrix for which ∑_n=0^∞Gⁿ <∞, then the stationary distribution of any stochastic matrix P≥ G can be expressed as a convex combination of the normalized rows of (I-G)^-1 = ∑_n=0^∞ Gⁿ. In this note, we give a short proof of this result that extends the theory to the countably infinite and continuous state space settings. This result plays an important role in obtaining error bounds in algorithms involving nearly decomposable Markov chains, and also in state truncations for Markov chains. We also use the representation to establish a new total variation distance error bound for truncated Markov chains.

Related

• On Convergence of a Truncation Scheme for Approximating Stationary Distributions of Continuous State Space Markov Chains and Processes
• Theoretical analysis of a Stochastic Approximation approach for computing Quasi-Stationary distributions of general state space Markov chains
• Approximation of Markov Chain Expectations and the Key Role of Stationary Distribution Convergence
• Limit formulas for the normalized fundamental matrix of the northwest-corner truncation of Markov chains: Matrix-infinite-product-form solutions of block-Hessenberg Markov chains
• Stationary distributions of continuous-time Markov chains: a review of theory and truncation-based approximations