Depth-Constrained ASV Navigation with Deep RL and Limited Sensing

2025/04/25 by Amirhossein Zhalehmehrabi, Zhalehmehrabi, Amirhossein, Daniele Meli +7
Earth and Planetary Sciences · Engineering · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Maritime Navigation and Safety #Robotics (cs.RO) #Underwater Acoustics Research #Underwater Vehicles and Communication Systems

paper · pdf · doi:10.48550/arxiv.2504.18253

openalex publication_date 2025/04/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Autonomous Surface Vehicles (ASVs) play a crucial role in maritime operations, yet their navigation in shallow-water environments remains challenging due to dynamic disturbances and depth constraints. Traditional navigation strategies struggle with limited sensor information, making safe and efficient operation difficult. In this paper, we propose a reinforcement learning (RL) framework for ASV navigation under depth constraints, where the vehicle must reach a target while avoiding unsafe areas with only a single depth measurement per timestep from a downward-facing Single Beam Echosounder (SBES). To enhance environmental awareness, we integrate Gaussian Process (GP) regression into the RL framework, enabling the agent to progressively estimate a bathymetric depth map from sparse sonar readings. This approach improves decision-making by providing a richer representation of the environment. Furthermore, we demonstrate effective sim-to-real transfer, ensuring that trained policies generalize well to real-world aquatic conditions. Experimental results validate our method's capability to improve ASV navigation performance while maintaining safety in challenging shallow-water environments.

Citations

• Nonlinear Model Predictive Control for Enhanced Navigation of Autonomous Surface Vessels
• Gaussian Process Regression under Computational and Epistemic Misspecification
• OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Research
• Memory-based Deep Reinforcement Learning for POMDPs
• Learning Quadrupedal Locomotion over Challenging Terrain
• Model-Reference Reinforcement Learning for Collision-Free Tracking Control of Autonomous Surface Vehicles
• Responsive Safety in Reinforcement Learning by PID Lagrangian Methods
• COLREG-Compliant Collision Avoidance for Unmanned Surface Vehicle using Deep Reinforcement Learning
• Taming an autonomous surface vehicle for path following and collision avoidance using deep reinforcement learning
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Control of a Quadrotor with Reinforcement Learning
• Constrained Policy Optimization
• Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates
• Adaptive Path Planning for Depth Constrained Bathymetric Mapping with an Autonomous Surface Vessel
• Deep Recurrent Q-Learning for Partially Observable MDPs
• End-to-End Training of Deep Visuomotor Policies

Related

• A Deep Reinforcement Learning Framework and Methodology for Reducing the Sim-to-Real Gap in ASV Navigation
• Depth Control of Model-Free AUVs via Reinforcement Learning
• How To Train Your HERON
• Distributional Reinforcement Learning based Integrated Decision Making and Control for Autonomous Surface Vehicles
• Digital Twin Supervised Reinforcement Learning Framework for Autonomous Underwater Navigation