Ergodic Theorems for discrete Markov chains

2017/07/27 by Halidias, Nikolaos
#60J10 #FOS: Mathematics #Probability (math.PR)

paper · doi:10.48550/arxiv.1707.08827

Abstract

Let X_n be a discrete time Markov chain with state space S (countably infinite, in general) and initial probability distribution μ⁽⁰⁾ = (P(X₀=i₁),P(X₀=i₂),⋯,). What is the probability of choosing in random some k ∈ ℕ with k ≤ n such that X_k = j where j ∈ S? This probability is the average (1)/(n) ∑_k=1ⁿ μ^(k)_j where μ^(k)_j = P(X_k = j). In this note we will study the limit of this average without assuming that the chain is irreducible, using elementary mathematical tools. Finally, we study the limit of the average (1)/(n) ∑_k=1ⁿ g(X_k) where g is a given function for a Markov chain not necessarily irreducible.

Related

• Perturbation theory and uniform ergodicity for discrete-time Markov chains
• Various proofs of the Fundamental Theorem of Markov Chains
• Average Behaviour in Discrete-Time Imprecise Markov Chains: A Study of\n Weak Ergodicity
• On the ergodicity of certain Markov chains in random environments
• Strict irreducibility of Markov chains and ergodicity of skew products