Existence and regularity of a nonhomogeneous transition matrix under measurability conditions

2008/04/28 by Liuer Ye, Ye, Liuer, Xianping Guo +3 · 1 citation
Mathematics · #60J27 #60J35 #60J75 #FOS: Mathematics #Probability (math.PR) #math.PR #msc:60J27 #msc:60J35 #msc:60J75

paper · pdf · doi:10.48550/arxiv.0804.4441

22 pages

arxiv created 2008/04/28 · arxiv updated 2009/12/01

Abstract

This paper is about the existence and regularity of the transition probability matrix of a nonhomogeneous continuous-time Markov process with a countable state space. A standard approach to prove the existence of such a transition matrix is to begin with a continuous (in t) and conservative matrix Q(t)=[q_ij(t)] of nonhomogeneous transition rates q_ij(t), and use it to construct the transition probability matrix. Here we obtain the same result except that the q_ij(t) are only required to satisfy a mild measurability condition, and Q(t) may not be conservative. Moreover, the resulting transition matrix is shown to be the minimum transition matrix and, in addition, a necessary and sufficient condition for it to be regular is obtained. These results are crucial in some applications of nonhomogeneous continuous-time Markov processes, such as stochastic optimal control problems and stochastic games, which motivated this work in the first place.

Cited by

• The Law of Large Numbers and CLT for Non-stationary Markov Jump Processes Exhibiting Time-of-Day Effects

Related

• Perturbations of continuous-time Markov chains
• An Imprecise Probabilistic Estimator for the Transition Rate Matrix of a\n Continuous-Time Markov Chain
• The Markov chain embedding problem in a low jump frequency context
• Estimating Transition Rates in Two-State Non-Homogeneous Markov Jump Processes with Intermittent Observations: A Pseudo-Marginal McMC Approach via Honest Times
• On solutions of Kolmogorov's equations for jump Markov processes