Learning and Retrieval from Prior Data for Skill-based Imitation Learning

2022/10/20 by Soroush Nasiriany, Nasiriany, Soroush, Tian Gao +5 · 21 citations
Computer Science · Engineering · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #Robot Manipulation and Learning #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.2210.11435

openalex publication_date 2022/10/20 · openalex created_date 2022/10/24 · openalex updated_date 2026/07/28

Abstract

Imitation learning offers a promising path for robots to learn general-purpose behaviors, but traditionally has exhibited limited scalability due to high data supervision requirements and brittle generalization. Inspired by recent advances in multi-task imitation learning, we investigate the use of prior data from previous tasks to facilitate learning novel tasks in a robust, data-efficient manner. To make effective use of the prior data, the robot must internalize knowledge from past experiences and contextualize this knowledge in novel tasks. To that end, we develop a skill-based imitation learning framework that extracts temporally extended sensorimotor skills from prior data and subsequently learns a policy for the target task that invokes these learned skills. We identify several key design choices that significantly improve performance on novel tasks, namely representation learning objectives to enable more predictable skill representations and a retrieval-based data augmentation mechanism to increase the scope of supervision for policy training. On a collection of simulated and real-world manipulation domains, we demonstrate that our method significantly outperforms existing imitation learning and offline reinforcement learning approaches. Videos and code are available at https://ut-austin-rpl.github.io/sailor

Cited by

• Open-Source Multimodal Moxin Models with Moxin-VLM and Moxin-VLA
• Try Once, Then Optimal: De-Redundified Procedure Memory for Cross-Episode Exploration Amortization
• Learning Skills from Action-Free Videos
• LoLA: Long Horizon Latent Action Learning for General Robot Manipulation
• OXE-AugE: A Large-Scale Robot Augmentation of OXE for Scaling Cross-Embodiment Policy Learning
• HiMoE-VLA: Hierarchical Mixture-of-Experts for Generalist Vision-Language-Action Policies
• ManualVLA: A Unified VLA Model for Chain-of-Thought Manual Generation and Robotic Manipulation
• LatBot: Distilling Universal Latent Actions for Vision-Language-Action Models
• NOIR 2.0: Neural Signal Operated Intelligent Robots for Everyday Activities
• Reinforcing Action Policies by Prophesying
• From Power to Precision: Learning Fine-grained Dexterity for Multi-fingered Robotic Hands
• Learning a Thousand Tasks in a Day
• TwinVLA: Data-Efficient Bimanual Manipulation with Twin Single-Arm Vision-Language-Action Models
• XR-1: Towards Versatile Vision-Language-Action Models via Learning Unified Vision-Motion Representations
• MLA: A Multisensory Language-Action Model for Multimodal Understanding and Forecasting in Robotic Manipulation
• Policy Compatible Skill Incremental Learning via Lazy Learning Interface
• Igniting VLMs toward the Embodied Space
• Data Retrieval with Importance Weights for Few-Shot Imitation Learning
• COLLAGE: Adaptive Fusion-based Retrieval for Augmented Policy Learning
• villa-X: Enhancing Latent Action Modeling in Vision-Language-Action Models
• Retrieve-Augmented Generation for Speeding up Diffusion Policy without Additional Training

Related

• Scalable Multi-Task Imitation Learning with Autonomous Improvement
• Object Manipulation Learning by Imitation
• Using Non-Expert Data to Robustify Imitation Learning via Offline Reinforcement Learning
• Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasets
• Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations