Real-world humanoid locomotion with reinforcement learning

2024/04/17 by Ilija Radosavovic, Tete Xiao, Bike Zhang +3 · 1 voice · 8 citations
Engineering · Computer Science · #Robotic Locomotion and Control #Reinforcement Learning in Robotics #Prosthetics and Rehabilitation Robotics

paper · doi:10.1126/scirobotics.adi9579

openalex publication_date 2024/04/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/30

Abstract

Humanoid robots that can autonomously operate in diverse environments have the potential to help address labor shortages in factories, assist elderly at home, and colonize new planets. Although classical controllers for humanoid robots have shown impressive results in a number of settings, they are challenging to generalize and adapt to new environments. Here, we present a fully learning-based approach for real-world humanoid locomotion. Our controller is a causal transformer that takes the history of proprioceptive observations and actions as input and predicts the next action. We hypothesized that the observation-action history contains useful information about the world that a powerful transformer model can use to adapt its behavior in context, without updating its weights. We trained our model with large-scale model-free reinforcement learning on an ensemble of randomized environments in simulation and deployed it to the real-world zero-shot. Our controller could walk over various outdoor terrains, was robust to external disturbances, and could adapt in context.

Citations

• Long Short-Term Memory
• Learning agile and dynamic motor skills for legged robots
• Learning dexterous in-hand manipulation
• Flamingo: a Visual Language Model for Few-Shot Learning

Cited by

• Shared Control of Holonomic Wheelchairs through Reinforcement Learning
• WITHDRAWN
• Robust RL Control for Bipedal Locomotion with Closed Kinematic Chains
• ULC: A Unified and Fine-Grained Controller for Humanoid Loco-Manipulation
• Learning Humanoid Arm Motion via Centroidal Momentum Regularized Multi-Agent Reinforcement Learning
• A Miniature High-Resolution Tension Sensor Based on a Photo-Reflector for Robotic Hands and Grippers
• Mechanical Intelligence-Aware Curriculum Reinforcement Learning for Humanoids with Parallel Actuation
• Learning Motion Skills with Adaptive Assistive Curriculum Force in Humanoid Robots

Discussions

• There are some cool examples of robots learning task-specific balance from scratch by evoalgs, reinforcement learning etc e.g. locomotion by rewarding distance travelled👇🏻 But then balance is bound [bsky, 0 points, 1 comments]