Reward Machine Inference for Robotic Manipulation

2024/12/13 by Mattijs Baert, Baert, Mattijs, Sam Leroux +3
Engineering · #Robot Manipulation and Learning

paper · pdf · doi:10.48550/arxiv.2412.10096

Abstract

Learning from Demonstrations (LfD) and Reinforcement Learning (RL) have\nenabled robot agents to accomplish complex tasks. Reward Machines (RMs) enhance\nRL's capability to train policies over extended time horizons by structuring\nhigh-level task information. In this work, we introduce a novel LfD approach\nfor learning RMs directly from visual demonstrations of robotic manipulation\ntasks. Unlike previous methods, our approach requires no predefined\npropositions or prior knowledge of the underlying sparse reward signals.\nInstead, it jointly learns the RM structure and identifies key high-level\nevents that drive transitions between RM states. We validate our method on\nvision-based manipulation tasks, showing that the inferred RM accurately\ncaptures task structure and enables an RL agent to effectively learn an optimal\npolicy.\n

Related

• Reinforcement learning with Demonstrations from Mismatched Task under Sparse Reward
• Unsupervised Perceptual Rewards for Imitation Learning
• Learning from Suboptimal Demonstration via Self-Supervised Reward Regression
• Residual Reinforcement Learning from Demonstrations
• Overcoming Exploration in Reinforcement Learning with Demonstrations