Deep reinforcement learning for turbulent drag reduction in channel flows

2023/01/24 by L. Guastoni, J. Rabault, Guastoni, L. +7 · 12 citations
Engineering · Physics and Astronomy · #FOS: Physical sciences #Fluid Dynamics (physics.flu-dyn) #Fluid Dynamics and Turbulent Flows #Model Reduction and Neural Networks #Traffic control and management

paper · pdf · doi:10.48550/arxiv.2301.09889

openalex publication_date 2023/01/24 · openalex created_date 2023/01/26 · openalex updated_date 2026/07/28

Abstract

We introduce a reinforcement learning (RL) environment to design and benchmark control strategies aimed at reducing drag in turbulent fluid flows enclosed in a channel. The environment provides a framework for computationally-efficient, parallelized, high-fidelity fluid simulations, ready to interface with established RL agent programming interfaces. This allows for both testing existing deep reinforcement learning (DRL) algorithms against a challenging task, and advancing our knowledge of a complex, turbulent physical system that has been a major topic of research for over two centuries, and remains, even today, the subject of many unanswered questions. The control is applied in the form of blowing and suction at the wall, while the observable state is configurable, allowing to choose different variables such as velocity and pressure, in different locations of the domain. Given the complex nonlinear nature of turbulent flows, the control strategies proposed so far in the literature are physically grounded, but too simple. DRL, by contrast, enables leveraging the high-dimensional data that can be sampled from flow simulations to design advanced control strategies. In an effort to establish a benchmark for testing data-driven control strategies, we compare opposition control, a state-of-the-art turbulence-control strategy from the literature, and a commonly-used DRL algorithm, deep deterministic policy gradient. Our results show that DRL leads to 43% and 46% drag reduction in a minimal and a larger channel (at a friction Reynolds number of 180), respectively, outperforming the classical opposition control by around 20 percentage points.

Cited by

• Shocks Under Control: Taming Transonic Compressible Flow over an RAE2822 Airfoil with Deep Reinforcement Learning
• Observer-based neural networks for flow estimation and control
• Navigation in a Three-Dimensional Urban Flow using Deep Reinforcement Learning
• Transfer learning strategies for accelerating reinforcement-learning-based flow control
• Physics-informed Neural-operator Predictive Control for Drag Reduction in Turbulent Flows
• Deep Reinforcement Learning in Action: Real-Time Control of Vortex-Induced Vibrations
• Real-time reinforcement learning for turbulent state-dependent control in a bluff-body wake
• Deep Reinforcement Learning for Active Flow Control around a Three-Dimensional Flow-Separated Wing at Re = 1,000
• Discovering Flow Separation Control Strategies in 3D Wings via Deep Reinforcement Learning
• Design of the wavy wall in a partially heated channel using CFD simulations and human-assisted Bayesian optimization
• Identifying efficient routes to laminarization: an optimization approach
• SmartFlow: A CFD-solver-agnostic deep reinforcement learning framework for computational fluid dynamics on HPC platforms

Related

• Drag-reduction strategies in wall-bounded turbulent flows using deep reinforcement learning
• Reinforcement learning of control strategies for reducing skin friction drag in a fully developed turbulent channel flow
• Deep reinforcement learning for flow control exploits different physics for increasing Reynolds-number regimes
• Physics-informed Neural-operator Predictive Control for Drag Reduction in Turbulent Flows
• Deep reinforcement learning for the management of the wall regeneration cycle in wall-bounded turbulent flows