ReinforceGen: Hybrid Skill Policies with Automated Data Generation and Reinforcement Learning

2025/12/18 by Zhou, Zihan, Garg, Animesh, Mandlekar, Ajay +1
Computer Science · Engineering · #Action (physics) #Adaptation (eye) #Artificial Intelligence (cs.AI) #Component (thermodynamics) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Motion (physics) #Motion planning #Reinforcement Learning in Robotics #Reinforcement learning #Reset (finance) #Robot Manipulation and Learning #Robotics #Robotics (cs.RO) #Soft Robotics and Applications #Task (project management)

paper · pdf · doi:10.48550/arxiv.2512.16861

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2025/12/18 · openalex created_date 2025/12/21 · openalex updated_date 2026/08/05

Abstract

Long-horizon manipulation has been a long-standing challenge in the robotics community. We propose ReinforceGen, a system that combines task decomposition, data generation, imitation learning, and motion planning to form an initial solution, and improves each component through reinforcement-learning-based fine-tuning. ReinforceGen first segments the task into multiple localized skills, which are connected through motion planning. The skills and motion planning targets are trained with imitation learning on a dataset generated from 10 human demonstrations, and then fine-tuned through online adaptation and reinforcement learning. When benchmarked on the Robosuite dataset, ReinforceGen reaches 80% success rate on all tasks with visuomotor controls in the highest reset range setting. Additional ablation studies show that our fine-tuning approaches contribute to an 89% average performance increase. Finally, ReinforceGen demonstrates significant improvement through fine-tuning in our real-world evaluations. More results and videos are available at https://reinforcegen.github.io.

Citations

• MoMaGen: Generating Demonstrations under Soft and Hard Constraints for Multi-Step Bimanual Mobile Manipulation
• Constraint-Preserving Data Generation for Visuomotor Policy Learning
• DemoGen: Synthetic Demonstration Generation for Data-Efficient Visuomotor Policy Learning
• DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning
• SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulation
• Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes
• OpenVLA: An Open-Source Vision-Language-Action Model
• Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Tasks
• FoundationPose-TensorRT
• NOD-TAMP: Generalizable Long-Horizon Planning with Neural Object Descriptors
• MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations
• Human-in-the-Loop Task and Motion Planning for Imitation Learning
• Language to Rewards for Robotic Skill Synthesis
• Imitating Task and Motion Planning with Visuomotor Transformers
• Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
• Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
• Diffusion policy: Visuomotor policy learning via action diffusion
• DayDreamer: World Models for Physical Robot Learning
• Behavior Transformers: Cloning k modes with one stone
• Guided Imitation of Task and Motion Planning
• Implicit Behavioral Cloning
• What Matters in Learning from Offline Human Demonstrations for Robot Manipulation
• Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning
• Integrated Task and Motion Planning
• Multi-Stage Learning for Grasp-Constrained Object Manipulation with a Simulated Panda Robot
• Residual Policy Learning
• Residual Reinforcement Learning for Robot Control
• Overcoming Exploration in Reinforcement Learning with Demonstrations
• #Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning
• Incentivizing Exploration In Reinforcement Learning With Deep Predictive Models

Related

• AW-Opt: Learning Robotic Skills with Imitation and Reinforcement at Scale
• SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulation
• Guided Imitation of Task and Motion Planning
• Residual Skill Policies: Learning an Adaptable Skill-based Action Space for Reinforcement Learning for Robotics
• Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learning