Cascade Markov Decision Processes: Theory and Applications

2015/09/01 by Manish Gupta, Gupta, Manish
Decision Sciences · Economics, Econometrics and Finance · #60J20 #90C40 #Decision-Making and Behavioral Economics #Economic theories and models #FOS: Electrical engineering #Stochastic processes and financial applications #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.1509.00392

openalex publication_date 2015/09/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper considers the optimal control of time varying continuous time Markov chains whose transition rates are themselves Markov processes. In one set of problems the solution of an ordinary differential equation is shown to determine the optimal performance and feedback controls, while some other cases are shown to lead to singular optimal control problems which are more difficult to solve. Solution techniques are demonstrated using examples from finance to behavioral decision making.

Citations

• Quantum Cat's Dilemma

Related

• Risk-sensitive control of continuous time Markov chains
• The existence of optimal control for continuous-time Markov decision processes in random environments
• Optimal Control of Markov Processes with Age-Dependent Transition Rates
• Continuous-Time Markov Decision Processes with Controlled Observations
• The existence of optimal feedback controls for stochastic dynamical systems with regime-switching