2013/09/28 by Ralf Banisch, Banisch, Ralf, Carsten Hartmann +1
Decision Sciences · Engineering · Mathematics · #65C05 #65C40 #65N30 #82C31 #Advanced Numerical Methods in Computational Mathematics #FOS: Mathematics #Markov Chains and Monte Carlo Methods #Numerical Analysis (math.NA) #Optimization and Control (math.OC) #Probabilistic and Robust Engineering Design
paper · pdf · doi:10.48550/arxiv.1309.7497
openalex publication_date 2013/09/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We propose a novel Galerkin discretization scheme for stochastic optimal control problems on an indefinite time horizon. The control problems are linear-quadratic in the controls, but possibly nonlinear in the state variables, and the discretization is based on the fact that problems of this kind can be transformed into linear boundary value problems by a logarithmic transformation. We show that the discretized linear problem is dual to a Markov decision problem, the precise form of which depends on the chosen Galerkin basis. We prove a strong error bound in L2 for the general scheme and discuss two special cases: a variant of the known Markov chain approximation obtained from a basis of characteristic functions of a box discretization, and a sparse approximation that uses the basis of committor functions of metastable sets of the dynamics; the latter is particularly suited for high-dimensional systems, e.g., control problems in molecular dynamics. We illustrate the method with several numerical examples, one being the optimal control of Alanine dipeptide to its helical conformation.