Learning Robust Controllers Via Probabilistic Model-Based Policy Search

2021/10/26 by Charvet, Valentin, Jensen, Bjørn Sand, Murray-Smith, Roderick
#FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2110.13576

Abstract

Model-based Reinforcement Learning estimates the true environment through a world model in order to approximate the optimal policy. This family of algorithms usually benefits from better sample efficiency than their model-free counterparts. We investigate whether controllers learned in such a way are robust and able to generalize under small perturbations of the environment. Our work is inspired by the PILCO algorithm, a method for probabilistic policy search. We show that enforcing a lower bound to the likelihood noise in the Gaussian Process dynamics model regularizes the policy updates and yields more robust controllers. We demonstrate the empirical benefits of our method in a simulation benchmark.

Related

• Model-based Policy Search for Partially Measurable Systems
• Robust Model-Based Reinforcement Learning with an Adversarial Auxiliary Model
• Fast Model-based Policy Search for Universal Policy Networks
• Reinforcement Learning Using Expectation Maximization Based Guided\n Policy Search for Stochastic Dynamics
• Model-Based Policy Gradients with Parameter-Based Exploration by Least-Squares Conditional Density Estimation