A model for system uncertainty in reinforcement learning

2018/02/21 by Murray, Ryan, Palladino, Michele
#FOS: Mathematics #Optimization and Control (math.OC)

paper · doi:10.48550/arxiv.1802.07668

Abstract

This work provides a rigorous framework for studying continuous time control problems in uncertain environments. The framework considered models uncertainty in state dynamics as a measure on the space of functions. This measure is considered to change over time as agents learn their environment. This model can be seem as a variant of either Bayesian reinforcement learning or adaptive control. We study necessary conditions for locally optimal trajectories within this model, in particular deriving an appropriate dynamic programming principle and Hamilton-Jacobi equations. This model provides one possible framework for studying the tradeoff between exploration and exploitation in reinforcement learning.

Related

• Direct Uncertainty Estimation in Reinforcement Learning
• Reinforcement Learning for Learning of Dynamical Systems in Uncertain Environment: a Tutorial
• Optimal Reinforcement Learning for Gaussian Systems
• Temporal Difference Uncertainties as a Signal for Exploration
• Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review