Using Neural Networks to Compute Approximate and Guaranteed Feasible Hamilton-Jacobi-Bellman PDE Solutions

2016/11/10 by Frank J. Jiang, Jiang, Frank, Glen Chou +5 · 4 citations
Computer Science · Physics and Astronomy · #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.1611.03158

openalex publication_date 2016/11/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

To sidestep the curse of dimensionality when computing solutions to Hamilton-Jacobi-Bellman partial differential equations (HJB PDE), we propose an algorithm that leverages a neural network to approximate the value function. We show that our final approximation of the value function generates near optimal controls which are guaranteed to successfully drive the system to a target state. Our framework is not dependent on state space discretization, leading to a significant reduction in computation time and space complexity in comparison with dynamic programming-based approaches. Using this grid-free approach also enables us to plan over longer time horizons with relatively little additional computation overhead. Unlike many previous neural network HJB PDE approximating formulations, our approximation is strictly conservative and hence any trajectories we generate will be strictly feasible. For demonstration, we specialize our new general framework to the Dubins car model and discuss how the framework can be applied to other models with higher-dimensional state spaces.

Citations

• Safe Platooning of Unmanned Aerial Vehicles via Reachability
• On Curves of Minimal Length with a Constraint on Average Curvature, and with Prescribed Initial and Terminal Positions and Tangents

Cited by

• Formal Safety Verification and Refinement for Generative Motion Planners via Certified Local Stabilization
• Parallel-Constraint Model Predictive Control: Exploiting Parallel Computation for Improving Safety
• Learning Robust Regions of Attraction Using Rollout-Enhanced Physics-Informed Neural Networks with Policy Iteration
• Hopf-type representation formulas and efficient algorithms for certain high-dimensional optimal control problems

Related

• Overcoming the curse of dimensionality for some Hamilton--Jacobi partial differential equations via neural network architectures
• Deep neural network approximation for high-dimensional parabolic Hamilton-Jacobi-Bellman equations
• Neural Actor-Critic Methods for Hamilton-Jacobi-Bellman PDEs: Asymptotic Analysis and Numerical Studies
• Approximation of Solution Operators for High-dimensional PDEs
• Recursive Regression with Neural Networks: Approximating the HJI PDE Solution