Contrastive Representation Learning for Robust Sim-to-Real Transfer of Adaptive Humanoid Locomotion

2025/09/16 by Lu, Yidan, Yang, Rurui, Kou, Qiran +5 · 1 citation
#FOS: Computer and information sciences #Robotics (cs.RO)

paper · doi:10.48550/arxiv.2509.12858

Abstract

Reinforcement learning has produced remarkable advances in humanoid locomotion, yet a fundamental dilemma persists for real-world deployment: policies must choose between the robustness of reactive proprioceptive control or the proactivity of complex, fragile perception-driven systems. This paper resolves this dilemma by introducing a paradigm that imbues a purely proprioceptive policy with proactive capabilities, achieving the foresight of perception without its deployment-time costs. Our core contribution is a contrastive learning framework that compels the actor's latent state to encode privileged environmental information from simulation. Crucially, this ``distilled awareness" empowers an adaptive gait clock, allowing the policy to proactively adjust its rhythm based on an inferred understanding of the terrain. This synergy resolves the classic trade-off between rigid, clocked gaits and unstable clock-free policies. We validate our approach with zero-shot sim-to-real transfer to a full-sized humanoid, demonstrating highly robust locomotion over challenging terrains, including 30 cm high steps and 26.5° slopes, proving the effectiveness of our method. Website: https://lu-yidan.github.io/cra-loco.

Citations

• A Unified and General Humanoid Whole-Body Controller for Versatile Locomotion
• Attention-Based Map Encoding for Learning Generalized Legged Locomotion
• MoRE: Mixture of Residual Experts for Humanoid Lifelike Gaits Learning on Complex Terrains
• Visual Imitation Enables Contextual Humanoid Control
• Learning Humanoid Locomotion with World Model Reconstruction
• BeamDojo: Learning Agile Humanoid Locomotion on Sparse Footholds
• Learning Humanoid Locomotion with Perceptive Internal Model
• Advancing Humanoid Locomotion: Mastering Challenging Terrains with Denoising World Model Learning
• Humanoid Parkour Learning
• Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Response
• Learning Vision-Based Bipedal Locomotion for Challenging Terrain
• Real-World Humanoid Locomotion with Reinforcement Learning
• DreamWaQ: Learning Robust Quadrupedal Locomotion With Implicit Terrain Imagination via Deep Reinforcement Learning
• Adapting Rapid Motor Adaptation for Bipedal Robots
• Learning Dynamic Bipedal Walking Across Stepping Stones
• Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning
• DeepWalk: Omnidirectional Bipedal Gait by Deep Reinforcement Learning
• Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning
• Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots
• Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward\n Composition
• Feedback Control For Cassie With Deep Reinforcement Learning
• Proximal Policy Optimization Algorithms
• Robust and Versatile Bipedal Jumping Control through Reinforcement Learning

Cited by

• Gait-Adaptive Perceptive Humanoid Locomotion with Real-Time Under-Base Terrain Reconstruction

Related

• Real-World Humanoid Locomotion with Reinforcement Learning
• PPF: Pre-training and Preservative Fine-tuning of Humanoid Locomotion via Model-Assumption-based Regularization
• MoRE: Mixture of Residual Experts for Humanoid Lifelike Gaits Learning on Complex Terrains
• Gait-Adaptive Perceptive Humanoid Locomotion with Real-Time Under-Base Terrain Reconstruction
• Advancing Humanoid Locomotion: Mastering Challenging Terrains with Denoising World Model Learning