Active Flow Control for Drag Reduction Through Multi-agent Reinforcement Learning on a Turbulent Cylinder at Re_D=3900

2025/03/05 by Pol Suárez, Francisco Alcántara-Ávila, Arnau Miró +4 · 1 voice · 1 citation
Engineering · Physics and Astronomy · #Fluid Dynamics and Turbulent Flows #Model Reduction and Neural Networks #Plasma and Flow Control in Aerodynamics

paper · pdf · doi:10.1007/s10494-025-00642-x

openalex publication_date 2025/03/05 · openalex created_date 2025/03/08 · openalex updated_date 2026/07/28

Abstract

Abstract This study presents novel drag reduction active-flow-control (AFC) strategies for a three-dimensional cylinder immersed in a flow at a Reynolds number based on freestream velocity and cylinder diameter of Re_D=3900 <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>R</mml:mi> <mml:msub> <mml:mi>e</mml:mi> <mml:mi>D</mml:mi> </mml:msub> <mml:mo>=</mml:mo> <mml:mn>3900</mml:mn> </mml:mrow> </mml:math> . The cylinder in this subcritical flow regime has been extensively studied in the literature and is considered a classic case of turbulent flow arising from a bluff body. The strategies presented are explored through the use of deep reinforcement learning. The cylinder is equipped with 10 independent zero-net-mass-flux jet pairs, distributed on the top and bottom surfaces, which define the AFC setup. The method is based on the coupling between a computational-fluid-dynamics solver and a multi-agent reinforcement-learning (MARL) framework using the proximal-policy-optimization algorithm. This work introduces a multi-stage training approach to expand the exploration space and enhance drag reduction stabilization. By accelerating training through the exploitation of local invariants with MARL, a drag reduction of approximately 9% <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mn>9</mml:mn> <mml:mo>%</mml:mo> </mml:mrow> </mml:math> is achieved. The cooperative closed-loop strategy developed by the agents is sophisticated, as it utilizes a wide bandwidth of mass-flow-rate frequencies, which classical control methods are unable to match. Notably, the mass cost efficiency is demonstrated to be two orders of magnitude lower than that of classical control methods reported in the literature. These developments represent a significant advancement in active flow control in turbulent regimes, critical for industrial applications.

Citations

• Closed-Loop Turbulence Control: Progress and Challenges
• On the identification of a vortex
• Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control
• How to train your robot with deep reinforcement learning: lessons we have learned

Cited by

• Deep Reinforcement Learning in Action: Real-Time Control of Vortex-Induced Vibrations

Discussions

• Another great output from our DRL for active flow control team! Lead by Pol Suarez, in this paper we show how DRL helps to achieve significant drag reduction for flow around a cylinder at Re=3900 with [bsky, 6 points, 0 comments]

Related

• Active flow control for drag reduction through multi-agent reinforcement learning on a turbulent cylinder at Re_D=3900
• Flow control of three-dimensional cylinders transitioning to turbulence via multi-agent reinforcement learning
• Active flow control for three-dimensional cylinders through deep reinforcement learning
• Active Flow Control for Bluff Body under High Reynolds Number Turbulent Flow Conditions Using Deep Reinforcement Learning
• Robust and Adaptive Deep Reinforcement Learning for Enhancing Flow Control around a Square Cylinder with Varying Reynolds Numbers