Coupling and a generalised Policy Iteration Algorithm in continuous time

2017/07/25 by Saul Jacka, Jacka, Saul D., Aleksandar Mijatović +3
Engineering · Mathematics · #49L99 #60H30 #93E20 #Advanced Control Systems Optimization #Carbon Dioxide Capture Technologies #FOS: Mathematics #Markov Chains and Monte Carlo Methods #Probability (math.PR)

paper · pdf · doi:10.48550/arxiv.1707.07834

openalex publication_date 2017/07/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We analyse a version of the policy iteration algorithm for the discounted infinite-horizon problem for controlled multidimensional diffusion processes, where both the drift and the diffusion coefficient can be controlled. We prove that, under assumptions on the problem data, the payoffs generated by the algorithm converge monotonically to the value function and an accumulation point of the sequence of policies is an optimal policy. The algorithm is stated and analysed in continuous time and state, with discretisation featuring neither in theorems nor the proofs. A key technical tool used to show that the algorithm is well-defined is the mirror coupling of Lindvall and Rogers.

Related

• Continuity of Cost in Borkar Control Topology and Implications on Discrete Space and Time Approximations for Controlled Diffusions under Several Criteria
• Discrete-Time Approximations of Controlled Diffusions with Infinite Horizon Discounted and Average Cost
• Exponential Lower Bounds For Policy Iteration
• A note on the policy iteration algorithm for discounted Markov decision processes for a class of semicontinuous models
• Approximate Policy Iteration Schemes: A Comparison