Efficient collective swimming by harnessing vortices through deep reinforcement learning

2018/02/07 by Siddhartha Verma, Guido Novati, Petros Koumoutsakos · 50 citations
Computer Science · Engineering · Environmental Science · Physics and Astronomy · #Aerospace engineering #Artificial intelligence #Biomimetic flight and propulsion mechanisms #Computer science #Economics #Energy (signal processing) #Engineering #Fidelity #Fish Ecology and Management Studies #Flow (mathematics) #Mechanics #Micro and Nano Robotics #Momentum (technical analysis) #Physics #Propulsion #Propulsive efficiency #Reinforcement Learning in Robotics #Reinforcement learning #Simulation #Vortex #cs.AI #physics.comp-ph #physics.flu-dyn

paper · pdf · open access · doi:10.1073/pnas.1800923115

published in Proceedings of the National Academy of Sciences 115(23), 5849-5854 (National Academy of Sciences) · 26 pages, 14 figures

arxiv created 2018/02/07 · openalex publication_date 2018/05/21 · arxiv updated 2022/06/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Fish in schooling formations navigate complex flow-fields replete with mechanical energy in the vortex wakes of their companions. Their schooling behaviour has been associated with evolutionary advantages including collective energy savings. How fish harvest energy from their complex fluid environment and the underlying physical mechanisms governing energy-extraction during collective swimming, is still unknown. Here we show that fish can improve their sustained propulsive efficiency by actively following, and judiciously intercepting, vortices in the wake of other swimmers. This swimming strategy leads to collective energy-savings and is revealed through the first ever combination of deep reinforcement learning with high-fidelity flow simulations. We find that a `smart-swimmer' can adapt its position and body deformation to synchronise with the momentum of the oncoming vortices, improving its average swimming-efficiency at no cost to the leader. The results show that fish may harvest energy deposited in vortices produced by their peers, and support the conjecture that swimming in formation is energetically advantageous. Moreover, this study demonstrates that deep reinforcement learning can produce navigation algorithms for complex flow-fields, with promising implications for energy savings in autonomous robotic swarms.

Cited by

• Toward Goal-Agnostic Joint-Embedding Predictive Control of Partial Differential Equations
• Learning to traverse convective flows at moderate to high Rayleigh numbers
• Physics-enhanced reinforcement learning for real-time optimal control of dynamical systems
• Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models
• Deep reinforcement learning for tracking a moving target in jellyfish-like swimming
• Accelerating high-order energy-stable discontinous Galerkin solver using auto-differentiation and neural networks
• Khalasi: Energy-Efficient Navigation for Surface Vehicles in Vortical Flow Fields
• How do trout regulate patterns of muscle contraction to optimize propulsive efficiency during steady swimming
• Optimal Undulatory Swimming with Constrained Deformation and Actuation Intervals
• Transfer learning strategies for accelerating reinforcement-learning-based flow control
• Deep Reinforcement Learning in Action: Real-Time Control of Vortex-Induced Vibrations
• Real-time reinforcement learning for turbulent state-dependent control in a bluff-body wake
• Attention on flow control: transformer-based reinforcement learning for lift regulation in highly disturbed flows
• Emergent interactions lead to collective frustration in robotic matter
• Optimizing Metachronal Paddling with Reinforcement Learning at Low Reynolds Number
• OGF: An Online Gradient Flow Method for Optimizing the Statistical Steady-State Time Averages of Unsteady Turbulent Flows
• Optimal Navigation in Microfluidics via the Optimization of a Discrete Loss
• Near Wake Dynamics of a Cross-Flow Turbine Array
• Data-assimilated model-informed reinforcement learning
• SpineWave: Harnessing Fish Rigid-Flexible Spinal Kinematics for Enhancing Biomimetic Robotic Locomotion
• Propulsion of a flexible foil in a wavy flow: resonance, antiresonance, and destructive self-interference
• Reinforcement Twinning for Hybrid Control of Flapping-Wing Drones
• Collective swimming pattern and synchronization of fish pairs (Gobiocypris rarus) in response to flow with different velocities
• A critical assessment of reinforcement learning methods for microswimmer navigation in complex flows
• Optimal Lattice Boltzmann Closures through Multi-Agent Reinforcement Learning
• Stable formations of self-propelled fish-like swimmers induced by hydrodynamic interactions. [europepmc]
• Learning from droplet flows in microfluidic channels using deep neural networks. [europepmc]
• On the energetics and stability of a minimal fish school. [europepmc]
• Control of chaotic systems by deep reinforcement learning. [europepmc]
• Flow Interactions Between Low Aspect Ratio Hydrofoils in In-line and Staggered Arrangements. [europepmc]
• Collective Behavior in Wild Zebrafish. [europepmc]
• Immersed Methods for Fluid-Structure Interaction. [europepmc]
• Reinforcement learning for bluff body active flow control in experiments and simulations. [europepmc]
• Vortex phase matching as a strategy for schooling in robots and in fish. [europepmc]
• A numerical study of fish adaption behaviors in complex environments with a deep reinforcement learning and immersed boundary-lattice Boltzmann method. [europepmc]
• Learning dominant physical processes with data-driven balance models. [europepmc]
• Tail Beat Synchronization during Schooling Requires a Functional Posterior Lateral Line System in Giant Danios, Devario aequipinnatus. [europepmc]
• Learning efficient navigation in vortical flow fields. [europepmc]
• Hydrodynamical Fingerprint of a Neighbour in a Fish Lateral Line. [europepmc]
• Molecular mechanisms of amyloid formation in living systems. [europepmc]
• An inertial mechanism behind dynamic station holding by fish swinging in a vortex street. [europepmc]
• Spontaneous vortex formation by microswimmers with retarded attractions. [europepmc]
• In-line swimming dynamics revealed by fish interacting with a robotic mechanism. [europepmc]
• Deep reinforcement learning for turbulent drag reduction in channel flows. [europepmc]
• Reinforcement learning for patient-specific optimal stenting of intracranial aneurysms. [europepmc]
• Learning to cooperate for low-Reynolds-number swimming: a model problem for gait coordination. [europepmc]
• Collective foraging of active particles trained by reinforcement learning. [europepmc]
• Energy conservation by collective movement in schooling fish. [europepmc]
• Molecular mechanism of α-synuclein aggregation on lipid membranes revealed. [europepmc]
• Mapping spatial patterns to energetic benefits in groups of flow-coupled swimmers. [europepmc]

Related

• Using Robotic Fish to Explore the Hydrodynamic Mechanism of Energy Saving in a Fish School
• The increased efficiency of fish swimming in a school
• Leader-follower formation enabled by pressure sensing in free-swimming undulatory robotic fish
• Model of collective fish behavior with hydrodynamic interactions
• Energy conservation by group dynamics in schooling fish