Nonuniqueness and Convergence to Equivalent Solutions in Observer-based Inverse Reinforcement Learning

2022/10/28 by Jared Town, Town, Jared, Zachary Morrison +3 · 1 citation
Computer Science · Engineering · #Adaptive Dynamic Programming Control #FOS: Computer and information sciences #FOS: Electrical engineering #Iterative Learning Control Systems #Machine Learning (cs.LG) #Piezoelectric Actuators and Control #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2210.16299

openalex publication_date 2022/10/28 · openalex created_date 2022/11/05 · openalex updated_date 2026/07/28

Abstract

A key challenge in solving the deterministic inverse reinforcement learning (IRL) problem online and in real-time is the existence of multiple solutions. Nonuniqueness necessitates the study of the notion of equivalent solutions, i.e., solutions that result in a different cost functional but same feedback matrix, and convergence to such solutions. While offline algorithms that result in convergence to equivalent solutions have been developed in the literature, online, real-time techniques that address nonuniqueness are not available. In this paper, a regularized history stack observer that converges to approximately equivalent solutions of the IRL problem is developed. Novel data-richness conditions are developed to facilitate the analysis and simulation results are provided to demonstrate the effectiveness of the developed technique.

Cited by

• A Convex Optimization Approach to Model-Free Inverse Optimal Control with Provable Convergence

Related

• Towards Resolving Unidentifiability in Inverse Reinforcement Learning
• Receding Horizon Inverse Reinforcement Learning
• Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matching
• Offline Inverse RL: New Solution Concepts and Provably Efficient Algorithms
• Adaptive Inverse Reinforcement Learning with Online Off-Policy Data Collection