Multi-Loco: Unifying Multi-Embodiment Legged Locomotion via Reinforcement Learning Augmented Diffusion

2025/06/13 by Shunpeng Yang, Zhen Fu, Yang, Shunpeng +11 · 2 citations
Computer Science · Engineering · #FOS: Computer and information sciences #Human Motion and Animation #Human Pose and Action Recognition #Robotic Locomotion and Control #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.2506.11470

openalex publication_date 2025/06/13 · openalex created_date 2025/10/11 · openalex updated_date 2026/07/28

Abstract

Generalizing locomotion policies across diverse legged robots with varying morphologies is a key challenge due to differences in observation/action dimensions and system dynamics. In this work, we propose Multi-Loco, a novel unified framework combining a morphology-agnostic generative diffusion model with a lightweight residual policy optimized via reinforcement learning (RL). The diffusion model captures morphology-invariant locomotion patterns from diverse cross-embodiment datasets, improving generalization and robustness. The residual policy is shared across all embodiments and refines the actions generated by the diffusion model, enhancing task-aware performance and robustness for real-world deployment. We evaluated our method with a rich library of four legged robots in both simulation and real-world experiments. Compared to a standard RL framework with PPO, our approach -- replacing the Gaussian policy with a diffusion model and residual term -- achieves a 10.35% average return improvement, with gains up to 13.57% in wheeled-biped locomotion tasks. These results highlight the benefits of cross-embodiment data and composite generative architectures in learning robust, generalized locomotion skills.

Citations

• BeamDojo: Learning Agile Humanoid Locomotion on Sparse Footholds
• Learning Humanoid Standing-up Control across Diverse Postures
• HOVER: Versatile Neural Whole-Body Controller for Humanoid Robots
• One Policy to Run Them All: an End-to-end Learning Approach to Multi-Embodiment Locomotion
• Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation
• Advancing Humanoid Locomotion: Mastering Challenging Terrains with Denoising World Model Learning
• OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning
• CleanDiffuser: An Easy-to-use Modularized Library for Diffusion Models in Decision Making
• Octo: An Open-Source Generalist Robot Policy
• CTS: Concurrent Teacher-Student Reinforcement Learning for Legged Locomotion
• DiffuseLoco: Real-Time Legged Locomotion Control with Diffusion from Offline Datasets
• ManyQuadrupeds: Learning a Single Locomotion Policy for Diverse Quadruped Robots
• Extreme Parkour with Legged Robots
• XSkill: Cross Embodiment Skill Discovery
• Polybot: Training One Policy Across Robots While Embracing Variability
• ANYmal Parkour: Learning Agile Navigation for Quadrupedal Robots
• Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
• Diffusion policy: Visuomotor policy learning via action diffusion
• Scalable Diffusion Models with Transformers
• GenLoco: Generalized Locomotion Controllers for Quadrupedal Robots
• DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps
• Elucidating the Design Space of Diffusion-Based Generative Models
• Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning
• Torch.manual_seed(3407) is all you need: On the influence of random seeds in deep learning architectures for computer vision
• Score-Based Generative Modeling through Stochastic Differential Equations
• Denoising Diffusion Implicit Models
• Learning agile and dynamic motor skills for legged robots

Cited by

• H-Zero: Cross-Humanoid Locomotion Pretraining Enables Few-shot Novel Embodiment Transfer
• Towards a Multi-Embodied Grasping Agent

Related

• One Policy to Run Them All: an End-to-end Learning Approach to Multi-Embodiment Locomotion
• DiffuseLoco: Real-Time Legged Locomotion Control with Diffusion from Offline Datasets
• Integrating Diffusion-based Multi-task Learning with Online Reinforcement Learning for Robust Quadruped Robot Control
• Transferable Latent-to-Latent Locomotion Policy for Efficient and Versatile Motion Control of Diverse Legged Robots
• Multi-Embodiment Locomotion at Scale with extreme Embodiment Randomization