2024/03/20 by Yue Yang, Yang, Yue, Bryce Ikeda +5
Computer Science · Engineering · #Augmented Reality Applications #FOS: Computer and information sciences #Graphics (cs.GR) #Machine Learning (cs.LG) #Robot Manipulation and Learning #Robotic Mechanisms and Dynamics #Robotics (cs.RO)
paper · pdf · doi:10.48550/arxiv.2403.13910
openalex publication_date 2024/03/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Robot Imitation Learning (IL) is a widely used method for training robots to perform manipulation tasks that involve mimicking human demonstrations to acquire skills. However, its practicality has been limited due to its requirement that users be trained in operating real robot arms to provide demonstrations. This paper presents an innovative solution: an Augmented Reality (AR)-assisted framework for demonstration collection, empowering non-roboticist users to produce demonstrations for robot IL using devices like the HoloLens 2. Our framework facilitates scalable and diverse demonstration collection for real-world tasks. We validate our approach with experiments on three classical robotics tasks: reach, push, and pick-and-place. The real robot performs each task successfully while replaying demonstrations collected via AR.