The Law of Large Numbers and CLT for Non-stationary Markov Jump Processes Exhibiting Time-of-Day Effects

2025/06/09 by Fischer, Monte, Glynn, Peter W.
#FOS: Mathematics #Probability (math.PR)

paper · doi:10.48550/arxiv.2506.08282

Abstract

In this paper, we develop a general law of large numbers and central limit theorem for cumulative reward processes associated with finite state Markov jump processes with non-stationary transition rates. Such models commonly arise in service operations and manufacturing applications in which time-of-day, day-of-week, and secular effects are of first-order importance in predicting system behavior. Our theorems allow for non-stationary reward environments that continuously accumulate reward, while also including contributions from non-stationary lump-sum rewards of random size that are collected at either jump times of the underlying process, jump times of a Poisson process modulated by the underlying process, or scheduled deterministic times. As part of our development, we also obtain a new central limit theorem for the special case in which the jump process transition rates and reward structure are periodic (as may occur over a weekly time interval), as well as for jump process models with resetting. We include a simulation study illustrating the quality of our CLT approximations for several non-stationary stochastic models.

Citations

• A Berry-Esseen theorem and Edgeworth expansions for uniformly elliptic inhomogeneous Markov chains
• Existence and regularity of a nonhomogeneous transition matrix under measurability conditions

Related

• Approximating Performance Measures for Slowly Changing Non-stationary Markov Chains
• The strong law of large numbers and a functional central limit theorem for general Markov additive processes
• A generalized integral fluctuation theorem for general jump processes
• A martingale approach to time-dependent and time-periodic linear response in Markov jump processes
• Jump Processes with Self-Interactions: Large Deviation Asymptotics