TossingBot: Learning to Throw Arbitrary Objects with Residual Physics

2019/03/27 by Andy Zeng, Shuran Song, Zeng, Andy +7 · 20 citations
Engineering · Neuroscience · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Motor Control and Adaptation #Muscle activation and electromyography studies #Robot Manipulation and Learning #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.1903.11239

openalex publication_date 2019/03/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We investigate whether a robot arm can learn to pick and throw arbitrary objects into selected boxes quickly and accurately. Throwing has the potential to increase the physical reachability and picking speed of a robot arm. However, precisely throwing arbitrary objects in unstructured settings presents many challenges: from acquiring reliable pre-throw conditions (e.g. initial pose of object in manipulator) to handling varying object-centric properties (e.g. mass distribution, friction, shape) and dynamics (e.g. aerodynamics). In this work, we propose an end-to-end formulation that jointly learns to infer control parameters for grasping and throwing motion primitives from visual observations (images of arbitrary objects in a bin) through trial and error. Within this formulation, we investigate the synergies between grasping and throwing (i.e., learning grasps that enable more accurate throws) and between simulation and deep learning (i.e., using deep networks to predict residuals on top of control parameters predicted by a physics simulator). The resulting system, TossingBot, is able to grasp and throw arbitrary objects into boxes located outside its maximum reach range at 500+ mean picks per hour (600+ grasps per hour with 85% throwing accuracy); and generalizes to new objects and target locations. Videos are available at https://tossingbot.cs.princeton.edu

Cited by

• From Overfitting to Reliability: Introducing the Hierarchical Approximate Bayesian Neural Network
• RoboTidy : A 3D Gaussian Splatting Household Tidying Benchmark for Embodied Navigation and Action
• Leveraging Neural Descriptor Fields for Learning Contact-Aware Dynamic Recovery
• Learning to Throw-Flip
• DA-MMP: Learning Coordinated and Accurate Throwing with Dynamics-Aware Motion Manifold Primitives
• Central Limit Theorems for Asynchronous Averaged Q-Learning
• DyDexHandover: Human-like Bimanual Dynamic Dexterous Handover using RGB-only Perception
• Near-Optimal Sample Complexity Bounds for Constrained Average-Reward MDPs
• Dynamics-Compliant Trajectory Diffusion for Super-Nominal Payload Manipulation
• Train-Once Plan-Anywhere Kinodynamic Motion Planning via Diffusion Trees
• Neural Robot Dynamics
• In-Context Iterative Policy Improvement for Dynamic Manipulation
• A Modular Residual Learning Framework to Enhance Model-Based Approach for Robust Locomotion
• AeroThrow: An Autonomous Aerial Throwing System for Precise Payload Delivery
• MoRE: Mixture of Residual Experts for Humanoid Lifelike Gaits Learning on Complex Terrains
• Hearing the Slide: Acoustic-Guided Constraint Learning for Fast Non-Prehensile Transport
• IMA-Catcher: An IMpact-Aware Nonprehensile Catching Framework based on Combined Optimization and Learning
• Learning Accurate Whole-body Throwing with High-frequency Residual Policy and Pullback Tube Acceleration
• SpikePingpong: Spike Vision-based Fast-Slow Pingpong Robot System
• ActivePusher: Active Learning and Planning with Residual Physics for Nonprehensile Manipulation

Related

• Harnessing the Synergy between Pushing, Grasping, and Throwing to Enhance Object Manipulation in Cluttered Scenarios
• Throwing Objects into A Moving Basket While Avoiding Obstacles
• Learning to Throw-Flip
• Data efficient Robotic Object Throwing with Model-Based Reinforcement Learning
• Learning Accurate Whole-body Throwing with High-frequency Residual Policy and Pullback Tube Acceleration