DyDexHandover: Human-like Bimanual Dynamic Dexterous Handover using RGB-only Perception

2025/09/22 by Haoran Zhou, Yangwei You, Zhou, Haoran +3
Computer Science · Engineering · Psychology · #FOS: Computer and information sciences #Hand Gesture Recognition Systems #Robot Manipulation and Learning #Robotics (cs.RO) #Social Robot Interaction and HRI

paper · pdf · doi:10.48550/arxiv.2509.17350

openalex publication_date 2025/09/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Dynamic in air handover is a fundamental challenge for dual-arm robots, requiring accurate perception, precise coordination, and natural motion. Prior methods often rely on dynamics models, strong priors, or depth sensing, limiting generalization and naturalness. We present DyDexHandover, a novel framework that employs multi-agent reinforcement learning to train an end to end RGB based policy for bimanual object throwing and catching. To achieve more human-like behavior, the throwing policy is guided by a human policy regularization scheme, encouraging fluid and natural motion, and enhancing the generalization capability of the policy. A dual arm simulation environment was built in Isaac Sim for experimental evaluation. DyDexHandover achieves nearly 99 percent success on training objects and 75 percent on unseen objects, while generating human-like throwing and catching behaviors. To our knowledge, it is the first method to realize dual-arm in-air handover using only raw RGB perception.

Citations

• Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids
• Learning Dexterous Bimanual Catch Skills through Adversarial-Cooperative Heterogeneous-Agent Reinforcement Learning
• Object-Centric Dexterous Manipulation from Human Motion Data
• Learning Diverse Bimanual Dexterous Manipulation Skills from Human Demonstrations
• Catch It! Learning to Catch in Flight with Mobile Dexterous Hands
• Dynamic Throwing with Robotic Material Handling Machines
• Geometric Fabrics: a Safe Guiding Medium for Policy Learning
• DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation
• DexCatch: Learning to Catch Arbitrary Objects with Dexterous Hands
• Dynamic Handover: Throw and Catch with Bimanual Hands
• A somato-cognitive action network alternates with effector regions in motor cortex
• DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation
• Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning
• Learning Generalizable Dexterous Manipulation from Human Grasp Affordance
• The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games
• TossingBot: Learning to Throw Arbitrary Objects with Residual Physics
• Learning Complex Dexterous Manipulation with Deep Reinforcement Learning\n and Demonstrations

Related

• Dynamic Handover: Throw and Catch with Bimanual Hands
• ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning
• DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Reality
• Bi-Manual Manipulation and Attachment via Sim-to-Real Reinforcement Learning
• DextrAH-RGB: Visuomotor Policies to Grasp Anything with Dexterous Hands