Optimal navigation of a smart active particle: directional and distance sensing

2023/11/17 by Putzke, Mischa, Stark, Holger
#500 Naturwissenschaften und Mathematik::530 Physik::530 Physik #Q learning #directional sensing #distance sensing #reinforcement learning #smart active particle

paper · doi:10.14279/depositonce-19484

Abstract

We employ Q learning, a variant of reinforcement learning, so that an active particle learns by itself to navigate on the fastest path toward a target while experiencing external forces and flow fields. As state variables, we use the distance and direction toward the target, and as action variables the active particle can choose a new orientation along which it moves with constant velocity. We explicitly investigate optimal navigation in a potential barrier/well and a uniform/ Poiseuille/swirling flow field. We show that Q learning is able to identify the fastest path and discuss the results. We also demonstrate that Q learning and applying the learned policy works when the particle orientation experiences thermal noise. However, the successful outcome strongly depends on the specific problem and the strength of noise.

Related

• Active particles using reinforcement learning to navigate in complex motility landscapes
• A critical assessment of reinforcement learning methods for microswimmer navigation in complex flows
• Smart navigation through a rotating barrier: Deep reinforcement learning with application to size-based separation of active microagents
• Optimal navigation in two-dimensional flows: Control theory and reinforcement learning
• Optimal navigation in a noisy environment