2017/09/22 by Mohit Sharma, Sharma, Mohit, Kris Kitani +3 · 1 citation
Business, Management and Accounting · Decision Sciences · #Artificial Intelligence (cs.AI) #Consumer Market Behavior and Pricing #FOS: Computer and information sciences #Innovation Diffusion and Forecasting #Supply Chain and Inventory Management
paper · pdf · doi:10.48550/arxiv.1709.07597
openalex publication_date 2017/09/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We make an important connection to existing results in econometrics to describe an alternative formulation of inverse reinforcement learning (IRL). In particular, we describe an algorithm using Conditional Choice Probabilities (CCP), which are maximum likelihood estimates of the policy estimated from expert demonstrations, to solve the IRL problem. Using the language of structural econometrics, we re-frame the optimal decision problem and introduce an alternative representation of value functions due to (Hotz and Miller 1993). In addition to presenting the theoretical connections that bridge the IRL literature between Economics and Robotics, the use of CCPs also has the practical benefit of reducing the computational cost of solving the IRL problem. Specifically, under the CCP representation, we show how one can avoid repeated calls to the dynamic programming subroutine typically used in IRL. We show via extensive experimentation on standard IRL benchmarks that CCP-IRL is able to outperform MaxEnt-IRL, with as much as a 5x speedup and without compromising on the quality of the recovered reward function.