Fast Policy Learning for 6-DOF Position Control of Underwater Vehicles

2025/12/15 by Sümer Tunçay, Alain Andrés, Tunçay, Sümer +4 · 1 voice
Computer Science · Engineering · #Adaptive Control of Nonlinear Systems #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #Robotics (cs.RO) #Underwater Vehicles and Communication Systems #cs.LG #cs.RO

paper · pdf · doi:10.48550/arxiv.2512.13359

openalex publication_date 2025/12/15 · arxiv published 2025/12/15 · openalex created_date 2025/12/17 · arxiv updated 2026/01/31 · openalex updated_date 2026/07/28

Abstract

Autonomous Underwater Vehicles (AUVs) require reliable six-degree-of-freedom (6-DOF) position control to operate effectively in complex and dynamic marine environments. Traditional controllers are effective under nominal conditions but exhibit degraded performance when faced with unmodeled dynamics or environmental disturbances. Reinforcement learning (RL) provides a powerful alternative but training is typically slow and sim-to-real transfer remains challenging. This work introduces a GPU accelerated RL training pipeline built in JAX and MuJoCo-XLA (MJX). By jointly JIT-compiling large-scale parallel physics simulation and learning updates, we achieve training times of under two minutes. Through systematic evaluation of multiple RL algorithms, we show robust 6-DOF trajectory tracking and effective disturbance rejection in real underwater experiments, with policies transferred zero-shot from simulation.

Citations

• MarineGym: A High-Performance Reinforcement Learning Platform for Underwater Robotics
• Learning to Swim: Reinforcement Learning for 6-DOF Control of Thruster-driven Autonomous Underwater Vehicles
• Accelerated Policy Learning with Parallel Differentiable Simulation
• Dropout Q-Functions for Doubly Efficient Reinforcement Learning
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• On the Continuity of Rotation Representations in Neural Networks
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Proximal Policy Optimization Algorithms
• Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
• CVXPY: A Python-Embedded Modeling Language for Convex Optimization
• Continuous control with deep reinforcement learning

Discussions

• Training RL for underwater robots used to take hours. We just changed that 🤖🌊 Policies trained in <5min, zero-shot deployed to real vehicles with precise 6-DoF control. Paper: arxiv.org/abs/2512.133 [bsky, 2 points, 0 comments]

Related

• Learning to Swim: Reinforcement Learning for 6-DOF Control of Thruster-driven Autonomous Underwater Vehicles
• MarineGym: Accelerated Training for Underwater Vehicles with High-Fidelity RL Simulation
• Toward 6-DOF Autonomous Underwater Vehicle Energy-Aware Position Control based on Deep Reinforcement Learning: Preliminary Results
• Scaling Multi Agent Reinforcement Learning for Underwater Acoustic Tracking via Autonomous Vehicles
• EasyUUV: An LLM-Enhanced Universal and Lightweight Sim-to-Real Reinforcement Learning Framework for UUV Attitude Control