2018/05/19 by Daniel S. Brown, Scott Niekum, Brown, Daniel S. +1
Computer Science · Decision Sciences · #Auction Theory and Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Reinforcement Learning in Robotics
paper · pdf · doi:10.48550/arxiv.1805.07687
openalex publication_date 2018/05/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Inverse reinforcement learning (IRL) infers a reward function from\ndemonstrations, allowing for policy improvement and generalization. However,\ndespite much recent interest in IRL, little work has been done to understand\nthe minimum set of demonstrations needed to teach a specific sequential\ndecision-making task. We formalize the problem of finding maximally informative\ndemonstrations for IRL as a machine teaching problem where the goal is to find\nthe minimum number of demonstrations needed to specify the reward equivalence\nclass of the demonstrator. We extend previous work on algorithmic teaching for\nsequential decision-making tasks by showing a reduction to the set cover\nproblem which enables an efficient approximation algorithm for determining the\nset of maximally-informative demonstrations. We apply our proposed machine\nteaching algorithm to two novel applications: providing a lower bound on the\nnumber of queries needed to learn a policy using active IRL and developing a\nnovel IRL algorithm that can learn more efficiently from informative\ndemonstrations than a standard IRL approach.\n