Difference of Convex Functions Programming Applied to Control with\n Expert Data

2016/06/03 by Bilal Piot, Matthieu Geist, Piot, Bilal +3
Computer Science · #Adaptive Dynamic Programming Control #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Optimization and Search Problems #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.1606.01128

openalex publication_date 2016/06/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper reports applications of Difference of Convex functions (DC)\nprogramming to Learning from Demonstrations (LfD) and Reinforcement Learning\n(RL) with expert data. This is made possible because the norm of the Optimal\nBellman Residual (OBR), which is at the heart of many RL and LfD algorithms, is\nDC. Improvement in performance is demonstrated on two specific algorithms,\nnamely Reward-regularized Classification for Apprenticeship Learning (RCAL) and\nReinforcement Learning with Expert Demonstrations (RLED), through experiments\non generic Markov Decision Processes (MDP), called Garnets.\n

Citations

• A Reduction of Imitation Learning and Structured Prediction to No-Regret\n Online Learning

Related

• Convex Programs and Lyapunov Functions for Reinforcement Learning: A Unified Perspective on the Analysis of Value-Based Methods
• Convex Q-Learning, Part 1: Deterministic Optimal Control
• Approximation of Convex Envelope Using Reinforcement Learning
• Inverse Reinforcement Learning via Convex Optimization
• On Connections between Constrained Optimization and Reinforcement Learning