2023/11/12 by Reich, Sebastian
Computer Science · Economics, Econometrics and Finance · Materials Science · #49L12 #65C35 #65M75 #93E20 #Catalytic Processes in Materials Science #FOS: Mathematics #Numerical Analysis (math.NA) #Optimization and Control (math.OC) #Stochastic Gradient Optimization Techniques #Stochastic processes and financial applications
paper · pdf · doi:10.48550/arxiv.2311.06906
openalex publication_date 2023/11/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The solution to a stochastic optimal control problem can be determined by computing the value function from a discretization of the associated Hamilton-Jacobi-Bellman equation. Alternatively, the problem can be reformulated in terms of a pair of forward-backward SDEs, which makes Monte-Carlo techniques applicable. More recently, the problem has also been viewed from the perspective of forward and reverse time SDEs and their associated Fokker-Planck equations. This approach is closely related to techniques used in diffusion-based generative models. Forward and reverse time formulations express the value function as the ratio of two probability density functions; one stemming from a forward McKean-Vlasov SDE and another one from a reverse McKean-Vlasov SDE. In this paper, we extend this approach to a more general class of stochastic optimal control problems and combine it with ensemble Kalman filter type and diffusion map approximation techniques in order to obtain efficient and robust particle-based algorithms.