Viscosity solutions approach to finite-horizon continuous-time Markov decision process

2020/03/31 by Zhong-Wei Liao, Jinghai Shao, Liao, Zhong-Wei +1
Economics, Econometrics and Finance · #60J27 #90C40 #93E20 #Climate Change Policy and Economics #FOS: Mathematics #Probability (math.PR)

paper · pdf · doi:10.48550/arxiv.2003.13982

openalex publication_date 2020/03/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/30

Abstract

This paper investigates the optimal control problems for the finite-horizon continuous-time Markov decision processes with delay-dependent control policies. We develop compactification methods in decision processes, and show that the existence of optimal policies. Subsequently, through the dynamic programming principle of the delay-dependent control policies, the differential-difference Hamilton-Jacobi-Bellman (HJB) equation in the setting of discrete space was established. Under certain conditions, we give the comparison principle and further prove that the value function is the unique viscosity solution to this HJB equation. Based on this, we show that among the class of delay-dependent control policies, there is an optimal one which is Markovian.

Related

• The existence of optimal control for continuous-time Markov decision processes in random environments
• Stochastic optimal control problems with delays in the state and in the control via viscosity solutions and applications to optimal advertising and optimal investment problems
• A Viscosity Approach to a Stochastic Control Problem on a Bounded Domain
• Impulsive control for continuous-time Markov Decision Processes
• Risk-sensitive control of continuous time Markov chains