Deep Reinforcement Learning for Online Control of Stochastic Partial Differential Equations

2021/10/21 by Erfan Pirmorad, Faraz Khoshbakhtian, Pirmorad, Erfan +5
Physics and Astronomy · Engineering · #Model Reduction and Neural Networks #Fluid Dynamics and Turbulent Flows #Traffic control and management

paper · pdf · doi:10.48550/arxiv.2110.11265

Abstract

In many areas, such as the physical sciences, life sciences, and finance, control approaches are used to achieve a desired goal in complex dynamical systems governed by differential equations. In this work we formulate the problem of controlling stochastic partial differential equations (SPDE) as a reinforcement learning problem. We present a learning-based, distributed control approach for online control of a system of SPDEs with high dimensional state-action space using deep deterministic policy gradient method. We tested the performance of our method on the problem of controlling the stochastic Burgers' equation, describing a turbulent fluid flow in an infinitely large domain.

Related

• Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learning
• Interpretable and Efficient Data-driven Discovery and Control of Distributed Systems
• Recent Developments in Machine Learning Methods for Stochastic Control and Games
• Deep BSDE-ML Learning and Its Application to Model-Free Optimal Control
• Reinforcement Learning with Function-Valued Action Spaces for Partial Differential Equation Control