Stochastic control up to a hitting time: optimality and rolling-horizon implementation

2008/06/18 by Debasish Chatterjee, Chatterjee, Debasish, Eugenio Cinquemani +4
Decision Sciences · Engineering · #90C39 #90C40 #93E20 #Advanced Control Systems Optimization #FOS: Mathematics #Optimization and Control (math.OC) #Probability (math.PR) #Process Optimization and Integration #Simulation Techniques and Applications

paper · pdf · doi:10.48550/arxiv.0806.3008

openalex publication_date 2008/06/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present a dynamic programming-based solution to a stochastic optimal control problem up to a hitting time for a discrete-time Markov control process. Firstly, we determine an optimal control policy to steer the process toward a compact target set while simultaneously minimizing an expected discounted cost. We then provide a rolling-horizon strategy for approximating the optimal policy, together with quantitative characterization of its sub-optimality with respect to the optimal policy. Finally, we address related issues of asymptotic discount-optimality of the value-iteration policy. Both the state and action spaces are assumed to be Polish.

Related

• Optimal Control Problem in a Stochastic Model with Periodic Hits on the Boundary of a Given Subset of the State Set (Tuning Problem)
• Optimal Control with Noisy Time
• Minimum-Time Stochastic Optimal Control Problems Under Mean Constraints and Application to Portfolio Investment
• Dynamic Programming Subject to Total Variation Distance Ambiguity
• A learning-based approach to stochastic optimal control under reach-avoid constraint