Solutions of Poisson's Equation for Stochastically Monotone Markov\n Chains

2022/02/21 by Peter W. Glynn, Glynn, Peter W., Alex Infanger +1
Business, Management and Accounting · Decision Sciences · Engineering · Social Sciences · #Advanced Queuing Theory Analysis #FOS: Mathematics #Probability (math.PR) #Probability and Risk Models #Simulation Techniques and Applications #Transportation Planning and Optimization #Transportation and Mobility Innovations

paper · pdf · doi:10.48550/arxiv.2202.10578

openalex publication_date 2022/02/21 · openalex created_date 2022/05/05 · openalex updated_date 2026/07/28

Abstract

Stochastically monotone Markov chains arise in many applied domains,\nespecially in the setting of queues and storage systems. Poisson's equation is\na key tool for analyzing additive functionals of such models, such as\ncumulative sums of waiting times or sums of rewards. In this paper, we show\nthat when the reward function for such a Markov chain is monotone, the solution\nof Poisson's equation is monotone. This implies that the value function\nassociated with infinite horizon average reward is monotone in the state when\nthe reward is monotone.\n

Related

• Specific bounds for a probabilistically interpretable solution of the Poisson equation for general state-space Markov chains with queueing applications
• Computable Bounds on the Solution to Poisson's Equation for General Harris Chains
• Solving Poisson's Equation: Existence, Uniqueness, Martingale Structure, and CLT
• A Note on the Stability of Monotone Markov Chains
• Poisson Hypothesis for information networks (A study in non-linear Markov processes)