Reinforcement-learning-based actuator selection method for active flow control

2023/01/12 by Romain Paris, Samir Beneddine, Julien Dandois · 4 citations
Engineering · Physics and Astronomy · #Fluid Dynamics and Turbulent Flows #Plasma and Flow Control in Aerodynamics #Model Reduction and Neural Networks

paper · doi:10.1017/jfm.2022.1043

Abstract

This paper addresses the issue of actuator selection for active flow control by proposing a novel method built on top of a reinforcement learning agent. Starting from a pre-trained agent using numerous actuators, the algorithm estimates the impact of a potential actuator removal on the value function, indicating the agent's performance. It is applied to two test cases, the one-dimensional Kuramoto–Sivashinsky equation and a laminar bidimensional flow around an airfoil at Re=1000 for different angles of attack ranging from 12^∘ to 20^∘ , to demonstrate its capabilities and limits. The proposed actuator-sparsification method relies on a sequential elimination of the least relevant action components, starting from a fully developed layout. The relevancy of each component is evaluated using metrics based on the value function. Results show that, while still being limited by this intrinsic elimination paradigm (i.e. the sequential elimination), actuator patterns and obtained policies demonstrate relevant performances and allow us to draw an accurate approximation of the Pareto front of performances versus actuator budget.

Citations

• Closed-Loop Turbulence Control: Progress and Challenges
• Post-stall flow control on an airfoil by local unsteady forcing
• Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control

Cited by

• Transfer learning strategies for accelerating reinforcement-learning-based flow control
• Sensor-Driven Surrogate Modeling and Control of Nonlinear Dynamical Systems Using FAE-CAE-LSTM and Deep Reinforcement Learning. [europepmc]
• Multi-agent Reinforcement Learning for the Control of Three-Dimensional Rayleigh-Bénard Convection. [europepmc]
• Reinforcement Operator Learning (ROL): A hybrid DeepONet-guided reinforcement learning framework for stabilizing the Kuramoto-Sivashinsky equation. [europepmc]

Related

• Reinforcement Learning-Based Closed-Loop Airfoil Flow Control
• Reinforcement Learning for Active Flow Control in Experiments
• Model Predictive and Reinforcement Learning Methods for Active Flow Control of an Airfoil with Dual-point Excitation of Plasma Actuators
• Optimizing Flow Control with Deep Reinforcement Learning: Plasma Actuator Placement around a Square Cylinder
• Deep-reinforcement-learning-based separation control in a two-dimensional airfoil