Minimax-Bayes Reinforcement Learning

2023/02/21 by Buening, Thomas Kleine, Dimitrakakis, Christos, Eriksson, Hannes +2
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.2302.10831

Abstract

While the Bayesian decision-theoretic framework offers an elegant solution to the problem of decision making under uncertainty, one question is how to appropriately select the prior distribution. One idea is to employ a worst-case prior. However, this is not as easy to specify in sequential decision making as in simple statistical estimation problems. This paper studies (sometimes approximate) minimax-Bayes solutions for various reinforcement learning problems to gain insights into the properties of the corresponding priors and policies. We find that while the worst-case prior depends on the setting, the corresponding minimax policies are more robust than those that assume a standard (i.e. uniform) prior.

Related

• Robust Bayesian reinforcement learning through tight lower bounds
• An Information-Theoretic Analysis of Bayesian Reinforcement Learning
• Bayesian Reinforcement Learning: A Survey
• Probabilistic Inference in Reinforcement Learning Done Right
• Information-Theoretic Minimax Regret Bounds for Reinforcement Learning based on Duality