Regularized Inverse Reinforcement Learning

2020/10/07 by Wonseok Jeon, Chen‐Yang Su, Jeon, Wonseok +9 · 3 citations
Computer Science · Decision Sciences · #Adaptive Dynamic Programming Control #Advanced Bandit Algorithms Research #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2010.03691

openalex publication_date 2020/10/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Inverse Reinforcement Learning (IRL) aims to facilitate a learner's ability to imitate expert behavior by acquiring reward functions that explain the expert's decisions. Regularized IRL applies strongly convex regularizers to the learner's policy in order to avoid the expert's behavior being rationalized by arbitrary constant rewards, also known as degenerate solutions. We propose tractable solutions, and practical methods to obtain them, for regularized IRL. Current methods are restricted to the maximum-entropy IRL framework, limiting them to Shannon-entropy regularizers, as well as proposing the solutions that are intractable in practice. We present theoretical backing for our proposed IRL method's applicability for both discrete and continuous controls, empirically validating our performance on a variety of tasks.

Citations

• GenDICE: Generalized Offline Estimation of Stationary Values

Cited by

• PersonaAgent with GraphRAG: Community-Aware Knowledge Graphs for Personalized LLM
• Offline Inverse Reinforcement Learning
• A Generalised Inverse Reinforcement Learning Framework

Related

• Convergence of a model-free entropy-regularized inverse reinforcement learning algorithm
• Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
• Inverse Reinforcement Learning via Convex Optimization
• Identifiability and Generalizability in Constrained Inverse Reinforcement Learning
• Statistical analysis of Inverse Entropy-regularized Reinforcement Learning