Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics

2025/01/17 by Li, Chenhao, Krause, Andreas, Hutter, Marco · 10 citations
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Robotics (cs.RO)

paper · doi:10.48550/arxiv.2501.10100

Abstract

Learning robust and generalizable world models is crucial for enabling efficient and scalable robotic control in real-world environments. In this work, we introduce a novel framework for learning world models that accurately capture complex, partially observable, and stochastic dynamics. The proposed method employs a dual-autoregressive mechanism and self-supervised training to achieve reliable long-horizon predictions without relying on domain-specific inductive biases, ensuring adaptability across diverse robotic tasks. We further propose a policy optimization framework that leverages world models for efficient training in imagined environments and seamless deployment in real-world systems. This work advances model-based reinforcement learning by addressing the challenges of long-horizon prediction, error accumulation, and sim-to-real transfer. By providing a scalable and robust framework, the introduced methods pave the way for adaptive and efficient robotic systems in real-world applications.

Cited by

• What about gravity in video generation? Post-Training Newton's Laws with Verifiable Rewards
• Target-Bench: Can Video World Models Achieve Mapless Path Planning with Semantic Targets?
• Learning Soft Robotic Dynamics with Active Exploration
• Efficient Model-Based Reinforcement Learning for Robot Control via Online Optimization
• A Comprehensive Survey on World Models for Embodied AI
• Pretraining in Actor-Critic Reinforcement Learning for Robot Locomotion
• Ego-Vision World Model for Humanoid Contact Planning
• Learning to Crawl: Latent Model-Based Reinforcement Learning for Soft Robotic Adaptive Locomotion
• VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulators
• Neural Robot Dynamics

Related

• Offline Robotic World Model: Learning Robotic Policies without a Physics Simulator
• WMPO: World Model-based Policy Optimization for Vision-Language-Action Models
• World4RL: Diffusion World Models for Policy Refinement with Reinforcement Learning for Robotic Manipulation
• Accelerating Model-Based Reinforcement Learning with State-Space World Models
• Enabling Efficient, Reliable Real-World Reinforcement Learning with Approximate Physics-Based Models