Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation
2024/01/04 by Zipeng Fu, Fu, Zipeng, Tony Z. Zhao +3 · 2 voices · 201 citations
Computer Science · Engineering · Medicine · Psychology · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Robot Manipulation and Learning #Robotics (cs.RO) #Social Robot Interaction and HRI #Stroke Rehabilitation and Recovery #Systems and Control (eess.SY) #cs.AI #cs.CV #cs.LG #cs.RO #eess.SY #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2401.02117
openalex publication_date 2024/01/04 · arxiv published 2024/01/04 · arxiv updated 2024/01/04 · openalex created_date 2024/01/08 · openalex updated_date 2026/07/28
Abstract
Imitation learning from human demonstrations has shown impressive performance in robotics. However, most results focus on table-top manipulation, lacking the mobility and dexterity necessary for generally useful tasks. In this work, we develop a system for imitating mobile manipulation tasks that are bimanual and require whole-body control. We first present Mobile ALOHA, a low-cost and whole-body teleoperation system for data collection. It augments the ALOHA system with a mobile base, and a whole-body teleoperation interface. Using data collected with Mobile ALOHA, we then perform supervised behavior cloning and find that co-training with existing static ALOHA datasets boosts performance on mobile manipulation tasks. With 50 demonstrations for each task, co-training can increase success rates by up to 90%, allowing Mobile ALOHA to autonomously complete complex mobile manipulation tasks such as sauteing and serving a piece of shrimp, opening a two-door wall cabinet to store heavy cooking pots, calling and entering an elevator, and lightly rinsing a used pan using a kitchen faucet. Project website: https://mobile-aloha.github.io
Cited by
- Imitation Learning for Robot Assistance in Open Surgery: A Multi-Policy Evaluation on Suture Following
- Vidarc: Embodied Video Diffusion Model for Closed-loop Control
- SAGA: Open-World Mobile Manipulation via Structured Affordance Grounding
- Robust Finetuning of Vision-Language-Action Robot Policies via Parameter Merging
- Sample from What You See: Visuomotor Policy Learning via Diffusion Bridge with Observation-Embedded Stochastic Differential Equation
- HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies
- Hybrid-Diffusion Models: Combining Open-loop Routines with Visuomotor Diffusion Policies
- FALCON: Actively Decoupled Visuomotor Policies for Loco-Manipulation with Foundation-Model-Based Coordination
- Video2Act: A Dual-System Video Diffusion Policy with Robotic Spatio-Motional Modeling
- Diagnose, Correct, and Learn from Manipulation Failures via Visual Symbols
- From Detection to Association: Learning Discriminative Object Embeddings for Multi-Object Tracking
- IGen: Scalable Data Generation for Robot Learning from Open-World Images
- ACE-F: A Cross Embodiment Foldable System with Force Feedback for Dexterous Teleoperation
- Compressor-VLA: Instruction-Guided Visual Token Compression for Efficient Robotic Manipulation
- SpeedAug: Policy Acceleration via Tempo-Enriched Policy and RL Fine-Tuning
- Unify Robot Actions in Camera Frame
- Bi-AQUA: Bilateral Control-Based Imitation Learning for Underwater Robot Arms via Lighting-Aware Action Chunking with Transformers
- VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulation
- AsyncVLA: Asynchronous Flow Matching for Vision-Language-Action Models
- SemanticVLA: Semantic-Aligned Sparsification and Enhancement for Efficient Robotic Manipulation
- Audio-VLA: Adding Contact Audio Perception to Vision-Language-Action Model for Robotic Manipulation
- RobustVLA: Robustness-Aware Reinforcement Post-Training for Vision-Language-Action Models
- A QP Framework for Improving Data Collection: Quantifying Device-Controller Performance in Robot Teleoperation
- 10 Open Challenges Steering the Future of Vision-Language-Action Models
- ViTaMIn-B: A Reliable and Efficient Visuo-Tactile Bimanual Manipulation Interface
- VLM-driven Skill Selection for Robotic Assembly Tasks
- TwinVLA: Data-Efficient Bimanual Manipulation with Twin Single-Arm Vision-Language-Action Models
- Embodiment Transfer Learning for Vision-Language-Action Models
- Temporal Action Selection for Action Chunking
- XR-1: Towards Versatile Vision-Language-Action Models via Learning Unified Vision-Motion Representations
- EgoMI: Learning Active Vision and Whole-Body Manipulation from Egocentric Human Demonstrations
- CheckVLA: Execution-Time Verification with Action-Conditioned World Model for Long-Horizon Mobile Manipulation
- UMI-on-Air: Embodiment-Aware Guidance for Embodiment-Agnostic Visuomotor Policies
- PACE: Phase-Aware Chunk Execution for Robot Policies with Action Chunking
- Blindfolded Experts Generalize Better: Insights from Robotic Manipulation and Videogames
- Awakening Facial Emotional Expressions in Human-Robot
- HRT1: One-Shot Human-to-Robot Trajectory Transfer for Mobile Manipulation
- FieldGen: From Teleoperated Pre-Manipulation Trajectories to Field-Guided Data Generation
- MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulation
- SoftMimic: Learning Compliant Whole-body Control from Examples
- Bridging Embodiment Gaps: Deploying Vision-Language-Action Models on Soft Robots
- Learning to Design Soft Hands using Reward Models
- RAPID Hand Prototype: Design of an Affordable, Fully-Actuated Biomimetic Hand for Dexterous Teleoperation
- Learning to Grasp Anything by Playing with Random Toys
- HiMaCon: Discovering Hierarchical Manipulation Concepts from Unlabeled Multi-Modal Data
- Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulation
- R2RGEN: Real-to-Real 3D Data Generation for Spatially Generalized Manipulation
- MiniBEE: A New Form Factor for Compact Bimanual Dexterity
- D2E: Scaling Vision-Action Pretraining on Desktop Data for Transfer to Embodied AI
- Prometheus: Universal, Open-Source Mocap-Based Teleoperation System with Force Feedback for Dataset Collection in Robot Learning
- Traj2Action: A Co-Denoising Framework for Trajectory-Guided Human-to-Robot Skill Transfer
- AIRoA MoMa Dataset: A Large-Scale Hierarchical Dataset for Mobile Manipulation
- VIVA+: Human-Centered Situational Decision-Making
- Control Your Robot: A Unified System for Robot Control and Policy Deployment
- GES-UniGrasp: A Two-Stage Dexterous Grasping Strategy With Geometry-Based Expert Selection
- FTACT: Force Torque aware Action Chunking Transformer for Pick-and-Reorient Bottle Task
- DemoGrasp: Universal Dexterous Grasping from a Single Demonstration
- AnywhereVLA: Language-Conditioned Exploration and Mobile Manipulation
- Parse-Augment-Distill: Learning Generalizable Bimanual Visuomotor Policies from Single Human Video
- DexDirect: Direct Kinesthetic Arm Guidance for Efficient Dexterous Demonstration Collection
- Critic Architecture Matters: Dual vs. Unified Critics for Humanoid Loco-Manipulation
- ROPA: Synthetic Robot Pose Generation for RGB-D Bimanual Data Augmentation
- Pure Vision Language Action (VLA) Models: A Comprehensive Survey
- Eva-VLA: Evaluating Vision-Language-Action Models' Robustness Under Real-World Physical Variations
- Bi-VLA: Bilateral Control-Based Imitation Learning via Vision-Language Fusion for Action Generation
- MV-UMI: A Scalable Multi-View Interface for Cross-Embodiment Learning
- N2M: Bridging Navigation and Manipulation by Learning Pose Preference from Rollout
- Learning Geometry-Aware Nonprehensile Pushing and Pulling with Dexterous Hands
- No Need for Real 3D: Fusing 2D Vision with Pseudo 3D Representations for Robotic Manipulation Learning
- M4Diffuser: Multi-View Diffusion Policy with Manipulability-Aware Control for Robust Mobile Manipulation
- SeqVLA: Sequential Task Execution for Long-Horizon Manipulation with Completion-Aware Vision-Language-Action Model
- Input-gated Bilateral Teleoperation: An Easy-to-implement Force Feedback Teleoperation Method for Low-cost Hardware
- RoboMatch: A Unified Mobile-Manipulation Teleoperation Platform with Auto-Matching Network Architecture for Long-Horizon Tasks
- FLOWER: Democratizing Generalist Robot Policies with Efficient Vision-Language-Action Flow Policies
- COMMET: A System for Human-Induced Conflicts in Mobile Manipulation of Everyday Tasks
- EMMA: Scaling Mobile Manipulation via Egocentric Human Data
- ANNIE: Be Careful of Your Robots
- Manipulation as in Simulation: Enabling Accurate Geometry Perception in Robots
- MoTo: A Zero-shot Plug-in Interaction-aware Navigation for General Mobile Manipulation
- Learning Fast, Tool aware Collision Avoidance for Collaborative Robots
- CogVLA: Cognition-Aligned Vision-Language-Action Model via Instruction-Driven Routing & Sparsification
- Mind and Motion Aligned: A Joint Evaluation IsaacSim Benchmark for Task Planning and Low-Level Policies in Mobile Manipulation
- HERMES: Human-to-Robot Embodied Learning from Multi-Source Motion Data for Mobile Dexterous Manipulation
- HyperTASR: Hypernetwork-Driven Task-Aware Scene Representations for Robust Manipulation
- Taming VR Teleoperation and Learning from Demonstration for Multi-Task Bimanual Table Service Manipulation
- Multimodal Data Storage and Retrieval for Embodied AI: A Survey
- Human Centric General Physical Intelligence for Agile Manufacturing Automation
- Actor-Critic for Continuous Action Chunks: A Reinforcement Learning Framework for Long-Horizon Robotic Manipulation with Sparse Reward
- 3D FlowMatch Actor: Unified 3D Policy for Single- and Dual-Arm Manipulation
- Large Model Empowered Embodied AI: A Survey on Decision-Making and Embodied Learning
- Leveraging OS-Level Primitives for Robotic Action Management
- GBC: Generalized Behavior-Cloning Framework for Whole-Body Humanoid Imitation
- Whole-Body Bilateral Teleoperation with Multi-Stage Object Parameter Estimation for Wheeled Humanoid Locomanipulation
- ODYSSEY: Open-World Quadrupeds Exploration and Manipulation for Long-Horizon Tasks
- AgentWorld: An Interactive Simulation Platform for Scene Construction and Mobile Robotic Manipulation
- Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation
- Constraint-Preserving Data Generation for Visuomotor Policy Learning
- TacMan-Turbo: Proactive Tactile Control for Robust and Efficient Articulated Object Manipulation
- Learning to Perform Low-Contact Autonomous Nasotracheal Intubation by Recurrent Action-Confidence Chunking with Transformer
- RoboMemory: A Brain-inspired Multi-memory Agentic Framework for Interactive Environmental Learning in Physical Embodied Systems
- COLLAGE: Adaptive Fusion-based Retrieval for Augmented Policy Learning
- Learning Pivoting Manipulation with Force and Vision Feedback Using Optimization-based Demonstrations
- On-Device Diffusion Transformer Policy for Efficient Robot Manipulation
- CHILD (Controller for Humanoid Imitation and Live Demonstration): a Whole-Body Humanoid Teleoperation System
- Exploring the Link Between Bayesian Inference and Embodied Intelligence: Toward Open Physical-World Embodied AI Systems
- Look, Focus, Act: Efficient and Robust Robot Learning via Human Gaze and Foveated Vision Transformers
- Ag2x2: Robust Agent-Agnostic Visual Representations for Zero-Shot Bimanual Manipulation
- Improving Low-Cost Teleoperation: Augmenting GELLO with Force
- EdgeVLA: Efficient Vision-Language-Action Models
- AnyPos: Automated Task-Agnostic Actions for Bimanual Manipulation
- Vidar: Embodied Video Diffusion Model for Generalist Manipulation
- The Developments and Challenges towards Dexterous and Embodied Robotic Manipulation: A Survey
- Diffusion-Based Imaginative Coordination for Bimanual Manipulation
- Object-Centric Mobile Manipulation through SAM2-Guided Perception and Imitation Learning
- Learning to Evaluate Autonomous Behaviour in Human-Robot Interaction
- TacPrint: A Wearable Fingertip Tactile Sensor for Human-to-Robot Contact Reproduction
- EmbodieDreamer: Advancing Real2Sim2Real Transfer for Policy Training via Embodied World Modeling
- DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge
- RwoR: Generating Robot Demonstrations from Human Hand Collection for Policy Learning without Robot
- TypeTele: Releasing Dexterity in Teleoperation by Dexterous Manipulation Types
- HumanoidGen: Data Generation for Bimanual Dexterous Manipulation via LLM Reasoning
- A Survey: Learning Embodied Intelligence from Physical Simulators and World Models
- Benchmarking Generalizable Bimanual Manipulation: RoboTwin Dual-Arm Collaboration Challenge at CVPR 2025 MEIS Workshop
- ParticleFormer: A 3D Point Cloud World Model for Multi-Object, Multi-Material Robotic Manipulation
- SEEA-R1: Tree-Structured Reinforcement Fine-Tuning for Self-Evolving Embodied Agents
- T-Rex: Task-Adaptive Spatial Representation Extraction for Robotic Manipulation with Vision-Language Models
- Haptic-Informed ACT with a Soft Gripper and Recovery-Informed Training for Pseudo Oocyte Manipulation
- RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
- Dex1B: Learning with 1B Demonstrations for Dexterous Manipulation
- CodeDiffuser: Attention-Enhanced Diffusion Policy via VLM-Generated Code for Instruction Ambiguity
- KARL: Kalman-Filter Assisted Reinforcement Learner for Dynamic Object Tracking and Grasping
- ControlVLA: Few-shot Object-centric Adaptation for Pre-trained Vision-Language-Action Models
- Vision in Action: Learning Active Perception from Human Demonstrations
- What Matters in Learning from Large-Scale Datasets for Robot Manipulation
- A Survey on Imitation Learning for Contact-Rich Tasks in Robotics
- AntiGrounding: Lifting Robotic Actions into VLM Representation Space for Decision Making
- SAIL: Faster-than-Demonstration Execution of Imitation Learning Policies
- ExoStart: Efficient learning for dexterous manipulation with sensorized exoskeleton demonstrations
- Eye, Robot: Learning to Look to Act with a BC-RL Perception-Action Loop
- GENMANIP: LLM-driven Simulation for Generalizable Instruction-Following Manipulation
- Gondola: Grounded Vision Language Planning for Generalizable Robotic Manipulation
- BiAssemble: Learning Collaborative Affordance for Bimanual Geometric Assembly
- LiPo: A Lightweight Post-optimization Framework for Smoothing Action Chunks Generated by Learned Policies
- DemoSpeedup: Accelerating Visuomotor Policies via Entropy-Guided Demonstration Acceleration
- AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Control
- SwitchVLA: Execution-Aware Task Switching for Vision-Language-Action Models
- SGN-CIRL: Scene Graph-based Navigation with Curriculum, Imitation, and Reinforcement Learning
- FreqPolicy: Frequency Autoregressive Visuomotor Policy with Continuous Tokens
- Fast-in-Slow: A Dual-System Foundation Model Unifying Fast Manipulation within Slow Reasoning
- FreeTacMan: Robot-free Visuo-Tactile Data Collection System for Contact-rich Manipulation
- HoMeR: Learning In-the-Wild Mobile Manipulation via Hybrid Imitation and Whole-Body Control
- SignBot: Learning Human-to-Humanoid Sign Language Interaction
- SCIZOR: A Self-Supervised Approach to Data Curation for Large-Scale Imitation Learning
- LabUtopia: High-Fidelity Simulation and Hierarchical Benchmark for Scientific Embodied Agents
- Spatial RoboGrasp: Generalized Robotic Grasping Control Policy
- Learning a Unified Policy for Position and Force Control in Legged Loco-Manipulation
- WorldEval: World Model as Real-World Robot Policies Evaluator
- ManiFeel: Benchmarking and Understanding Visuotactile Manipulation Policy Learning
- Genie Centurion: Accelerating Scalable Real-World Robot Training with Human Rewind-and-Refine Guidance
- Canonical Policy: Learning Canonical 3D Representation for SE(3)-Equivariant Policy
- Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills
- Bootstrapping Imitation Learning for Long-horizon Manipulation via Hierarchical Data Collection Space
- Object-Focus Actor for Data-efficient Robot Generalization Dexterous Manipulation
- TeleOpBench: A Simulator-Centric Benchmark for Dual-Arm Dexterous Teleoperation
- Motion Planning for Mobile Manipulators Navigating Doorways via Model Predictive Control
- Latent Theory of Mind: A Decentralized Diffusion Architecture for Cooperative Manipulation
- Augmented Reality for RObots (ARRO): Pointing Visuomotor Policies Towards Visual Robustness
- Extracting Visual Plans from Unlabeled Videos via Symbolic Guidance
- UniSkill: Imitating Human Videos via Cross-Embodiment Skill Representations
- D-CODA: Diffusion for Coordinated Dual-Arm Data Augmentation
- SkillMimic-V2: Learning Robust and Generalizable Interaction Skills from Sparse and Noisy Demonstrations
- DexCtrl: Towards Sim-to-Real Dexterity with Adaptive Controller Learning
- Superhuman Safe and Agile Racing through Multi-Agent Reinforcement Learning
- B-spline Policy: Accelerating Manipulation Policies via B-spline Action Representations
- Towards the Vision-Sound-Language-Action Paradigm: The HEAR Framework for Sound-Centric Manipulation
- A Survey of Robotic Navigation and Manipulation with Physics Simulators in the Era of Embodied AI
- RoboPocket: Improve Robot Policies Instantly with Your Phone
- Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments
- Motion Generation for Food Topping Challenge 2024: Serving Salmon Roe Bowl and Picking Fried Chicken
- UTTG_ A Universal Teleoperation Approach via Online Trajectory Generation
- Demonstrating DVS: Dynamic Virtual-Real Simulation Platform for Mobile Robotic Tasks
- STDArm: Transferring Visuomotor Policies From Static Data Training to Dynamic Robot Manipulation
- Dynamic Camera Poses and Where to Find Them
- Integrating Learning-Based Manipulation and Physics-Based Locomotion for Whole-Body Badminton Robot Control
- MobileWAM: Bridging World Action Models to Mobile Manipulation with Chain-of-Foresight
- Geometric Formulation of Unified Force-Impedance Control on SE(3) for Robotic Manipulators
- Physically Consistent Humanoid Loco-Manipulation using Latent Diffusion Models
- Tool-as-Interface: Learning Robot Policies from Observing Human Tool Use
- Few-Shot Vision-Language Action-Incremental Policy Learning
- SuFIA-BC: Generating High Quality Demonstration Data for Visuomotor Policy Learning in Surgical Subtasks
- Dexterous Manipulation through Imitation Learning: A Survey
- DiffOG: Differentiable Policy Trajectory Optimization with Generalizability
- RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins
- Krysalis Hand: A Lightweight, High-Payload, 18-DoF Anthropomorphic End-Effector for Robotic Learning and Dexterous Manipulation
- World-to-Wrist: Task-Conditioned Future Wrist Modeling for Fine-Grained Robot Manipulation
- HUMOTO: A 4D Dataset of Mocap Human Object Interactions
- Flying Hand: End-Effector-Centric Framework for Versatile Aerial Manipulation Teleoperation and Policy Learning
- Efficient Task-specific Conditional Diffusion Policies: Shortcut Model Acceleration and SO(3) Optimization
- Echo: An Open-Source, Low-Cost Teleoperation System with Force Feedback for Dataset Collection in Robot Learning
- TOCALib: Optimal control library with interpolation for bimanual manipulation and obstacles avoidance
- ViTaMIn: Learning Contact-Rich Tasks Through Robot-Free Visuo-Tactile Manipulation Interface
Discussions
Related