Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control

2019/02/20 by Jean Rabault, Miroslav Kuchta, Atle Jensen +2 · 35 citations
Physics and Astronomy · Engineering · #Model Reduction and Neural Networks #Fluid Dynamics and Turbulent Flows #Fluid Dynamics and Vibration Analysis

paper · doi:10.1017/jfm.2019.62

Abstract

We present the first application of an artificial neural network trained through a deep reinforcement learning agent to perform active flow control. It is shown that, in a two-dimensional simulation of the Kármán vortex street at moderate Reynolds number ( Re=100 ), our artificial neural network is able to learn an active control strategy from experimenting with the mass flow rates of two jets on the sides of a cylinder. By interacting with the unsteady wake, the artificial neural network successfully stabilizes the vortex alley and reduces drag by approximately 8 %. This is performed while using small mass flow rates for the actuation, of the order of 0.5 % of the mass flow rate intersecting the cylinder cross-section once a new pseudo-periodic shedding regime is found. This opens the way to a new class of methods for performing active flow control.

Cited by

• Dream training for active control of multiphase flow in spray fluidized beds
• Reinforcement learning of control strategies for reducing skin friction drag in a fully developed turbulent channel flow
• Reinforcement-learning-based actuator selection method for active flow control
• Learn to flap: foil non-parametric path planning via deep reinforcement learning
• Deep reinforcement learning finds a new strategy for vortex-induced vibration control
• Physics of the vortex gust–airfoil interaction under an optimal mitigation strategy learned through deep reinforcement learning
• Robust flow control and optimal sensor placement using deep reinforcement learning
• Navigation in a simplified urban flow through deep reinforcement learning
• Direct shape optimization through deep reinforcement learning
• Dynamic feature-based deep reinforcement learning for flow control of circular cylinder with sparse surface pressure sensing
• Machine Learning for Fluid Mechanics
• Toward Goal-Agnostic Joint-Embedding Predictive Control of Partial Differential Equations
• Physics-enhanced reinforcement learning for real-time optimal control of dynamical systems
• Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models
• Reinforcement Learning for Active Flow Control in Experiments
• WaterLily.jl: A differentiable and backend-agnostic Julia solver for incompressible viscous flow around dynamic bodies
• Deep Reinforcement Learning for Active Flow Control around a Circular Cylinder Using Unsteady-mode Plasma Actuators
• Using reinforcement learning to probe the role of feedback in skill acquisition
• Extreme Aerodynamics: A Data-Driven Perspective
• Learning to Control PDEs with Differentiable Predictive Control and Time-Integrated Neural Operators
• Shocks Under Control: Taming Transonic Compressible Flow over an RAE2822 Airfoil with Deep Reinforcement Learning
• Modelling of Flow Past Long Cylindrical Structures
• Observer-based neural networks for flow estimation and control
• Active Flow Control for Drag Reduction Through Multi-agent Reinforcement Learning on a Turbulent Cylinder at Re_D=3900
• Transfer learning strategies for accelerating reinforcement-learning-based flow control
• Active Control of Turbulent Airfoil Flows Using Adjoint-based Deep Learning
• Physics-informed Neural-operator Predictive Control for Drag Reduction in Turbulent Flows
• Deep Reinforcement Learning in Action: Real-Time Control of Vortex-Induced Vibrations
• Routes towards an effective AI in CFD: an epistemological and technical perspective
• Real-time reinforcement learning for turbulent state-dependent control in a bluff-body wake
• Discovering Flow Separation Control Strategies in 3D Wings via Deep Reinforcement Learning
• Machine learning in fluid dynamics: A critical assessment
• Identifying efficient routes to laminarization: an optimization approach
• SmartFlow: A CFD-solver-agnostic deep reinforcement learning framework for computational fluid dynamics on HPC platforms
• Towards human-interpretable, automated learning of feedback control for the mixing layer

Related

• Deep reinforcement learning for flow control exploits different physics for increasing Reynolds-number regimes
• Deep Reinforcement Learning achieves flow control of the 2D Karman Vortex Street
• Deep Reinforcement Learning for Active Flow Control around a Circular Cylinder Using Unsteady-mode Plasma Actuators
• Active Flow Control for Bluff Body under High Reynolds Number Turbulent Flow Conditions Using Deep Reinforcement Learning
• Active Control of Flow over Rotating Cylinder by Multiple Jets using Deep Reinforcement Learning