RoboCLIP: One Demonstration is Enough to Learn Robot Policies

2023/10/11 by Sumedh Sontakke, Sontakke, Sumedh A, Jesse Zhang +13 · 22 citations
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Reinforcement Learning in Robotics #Robot Manipulation and Learning #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.2310.07899

openalex publication_date 2023/10/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Reward specification is a notoriously difficult problem in reinforcement learning, requiring extensive expert supervision to design robust reward functions. Imitation learning (IL) methods attempt to circumvent these problems by utilizing expert demonstrations but typically require a large number of in-domain expert demonstrations. Inspired by advances in the field of Video-and-Language Models (VLMs), we present RoboCLIP, an online imitation learning method that uses a single demonstration (overcoming the large data requirement) in the form of a video demonstration or a textual description of the task to generate rewards without manual reward function design. Additionally, RoboCLIP can also utilize out-of-domain demonstrations, like videos of humans solving the task for reward generation, circumventing the need to have the same demonstration and deployment domains. RoboCLIP utilizes pretrained VLMs without any finetuning for reward generation. Reinforcement learning agents trained with RoboCLIP rewards demonstrate 2-3 times higher zero-shot performance than competing imitation learning methods on downstream robot manipulation tasks, doing so using only one video/text demonstration.

Cited by

• Embodied Learning of Reward for Musculoskeletal Control with Vision Language Models
• EVOLVE-VLA: Test-Time Training from Environment Feedback for Vision-Language-Action Models
• An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges
• SIMA 2: A Generalist Embodied Agent for Virtual Worlds
• BiTAgent: A Task-Aware Modular Framework for Bidirectional Coupling between Multimodal Large Language Models and World Models
• Goal-Driven Reward by Video Diffusion Models for Reinforcement Learning
• LLM-Driven Stationarity-Aware Expert Demonstrations for Multi-Agent Reinforcement Learning in Mobile Systems
• AutoFocus-IL: VLM-based Saliency Maps for Data-Efficient Visual Imitation Learning without Extra Human Annotations
• π^*_0.6: a VLA That Learns From Experience
• PROF: An LLM-based Reward Code Preference Optimization Framework for Offline Imitation Learning
• Automated Reward Design for Gran Turismo
• Reflection-Based Task Adaptation for Self-Improving VLA
• CDE: Concept-Driven Exploration for Reinforcement Learning
• From Data to Rewards: a Bilevel Optimization Perspective on Maximum Likelihood Estimation
• DexFlyWheel: A Scalable and Self-improving Data Generation Framework for Dexterous Manipulation
• LAGEA: Language Guided Embodied Agents for Robotic Manipulation
• ReLAM: Learning Anticipation Model for Rewarding Visual Robotic Manipulation
• DesignCLIP: Multimodal Learning with CLIP for Design Patent Understanding
• Policy Learning from Large Vision-Language Model Feedback without Reward Modeling
• FOUNDER: Grounding Foundation Models in World Models for Open-Ended Embodied Decision Making
• Foundation Model Driven Robotics: A Comprehensive Review
• GoalLadder: Incremental Goal Discovery with Vision-Language Models

Related

• AW-Opt: Learning Robotic Skills with Imitation and Reinforcement at Scale
• Learning Manipulation Skills through Robot Chain-of-Thought with Sparse Failure Guidance
• A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges
• Imitation Bootstrapped Reinforcement Learning
• RoboManipBaselines: A Unified Framework for Imitation Learning in Robotic Manipulation across Real and Simulated Environments