Learning Stabilizing Policies in Stochastic Control Systems

2022/05/24 by Đorđe Žikelić, Žikelić, Đorđe, Mathias Lechner +5
Computer Science · Physics and Astronomy · #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Optimization and Control (math.OC) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2205.11991

openalex publication_date 2022/05/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this work, we address the problem of learning provably stable neural network policies for stochastic control systems. While recent work has demonstrated the feasibility of certifying given policies using martingale theory, the problem of how to learn such policies is little explored. Here, we study the effectiveness of jointly learning a policy together with a martingale certificate that proves its stability using a single learning algorithm. We observe that the joint optimization problem becomes easily stuck in local minima when starting from a randomly initialized policy. Our results suggest that some form of pre-training of the policy is required for the joint optimization to repair and verify the policy successfully.

Related

• Learning Provably Stabilizing Neural Controllers for Discrete-Time Stochastic Systems
• Stochastic Reinforcement Learning with Stability Guarantees for Control of Unknown Nonlinear Systems
• Stability Verification in Stochastic Control Systems via Neural Network Supermartingales
• An Efficient On-Policy Deep Learning Framework for Stochastic Optimal Control
• Learning Control Policies for Stochastic Systems with Reach-avoid Guarantees