2005/05/31 by Hilbert J. Kappen, H. J. Kappen · 322 citations
Economics, Econometrics and Finance · Mathematics · Physics and Astronomy · #Advanced Thermodynamics and Statistical Mechanics #Applied mathematics #Laplace transform #Mathematical analysis #Mathematical optimization #Mathematics #Model Reduction and Neural Networks #Optimal control #Path integral formulation #Physics #Quantum mechanics #Stochastic control #Stochastic processes and financial applications #physics.comp-ph #physics.gen-ph
paper · pdf · doi:10.1088/1742-5468/2005/11/p11011
published in Journal of Statistical Mechanics Theory and Experiment 2005(11), P11011 (Institute of Physics) · 21 pages, 6 figures, submitted to JSTAT
arxiv created 2005/10/07 · openalex publication_date 2005/11/01 · arxiv updated 2009/12/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/08
This paper considers linear-quadratic control of a non-linear dynamical system subject to arbitrary cost. I show that for this class of stochastic control problems the non-linear Hamilton–Jacobi–Bellman equation can be transformed into a linear equation. The transformation is similar to the transformation used to relate the classical Hamilton–Jacobi equation to the Schrödinger equation. As a result of the linearity, the usual backward computation can be replaced by a forward diffusion process that can be computed by stochastic integration or by the evaluation of a path integral. It is shown how in the deterministic limit the Pontryagin minimum principle formalism is recovered. The significance of the path integral approach is that it forms the basis for a number of efficient computational methods, such as Monte Carlo sampling, the Laplace approximation and the variational approximation. We show the effectiveness of the first two methods in a number of examples. Examples are given that show the qualitative difference between stochastic and deterministic control and the occurrence of symmetry breaking as a function of the noise.