A Uniform-grid Discretization Algorithm for Stochastic Control with Risk\n Constraints

2015/01/08 by Yinlam Chow, Marco Pavone, Chow, Yin-Lam +1
Decision Sciences · Economics, Econometrics and Finance · #Risk and Portfolio Optimization #Economic theories and models #Stochastic processes and financial applications

paper · pdf · doi:10.48550/arxiv.1501.02024

Abstract

In this paper, we present a discretization algorithm for finite horizon risk\nconstrained dynamic programming algorithm in [Chow_Pavone₁3]. Although in a\ntheoretical standpoint, Bellman's recursion provides a systematic way to find\noptimal value functions and generate optimal history dependent policies, there\nis a serious computational issue. Even if the state space and action space of\nthis constrained stochastic optimal control problem are finite, the spaces of\nrisk threshold and the feasible risk update are closed bounded subset of real\nnumbers. This prohibits any direct applications of unconstrained finite state\niterative methods in dynamic programming found in [Bertsekas₀5]. In order to\napproximate Bellman's operator derived in [Chow_Pavone₁3], we discretize the\ncontinuous action spaces and formulate a finite space approximation for the\nexact dynamic programming algorithm. We will also prove that the approximation\nerror bound of optimal value functions is bound linearly by the step size of\ndiscretization. Finally, details for implementations and possible modifications\nare discussed.\n

Related

• Stochastic Optimal Control With Dynamic, Time-Consistent Risk Constraints
• Dynamic programming and dimensionality in convex stochastic optimization and control
• Dynamic Programming Subject to Total Variation Distance Ambiguity
• Nonlinear Monte Carlo methods with polynomial runtime for Bellman equations of discrete time high-dimensional stochastic optimal control problems
• Stochastic viability and dynamic programming