2017/03/21 by Konstantinos Chatzilygeroudis, Chatzilygeroudis, Konstantinos, Roberto Rama +9 · 2 citations
Computer Science · Mathematics · #Artificial intelligence #Benchmark (surveying) #Black box #Computer science #Constraint (computer-aided design) #FOS: Computer and information sciences #Flexibility (engineering) #Function (biology) #Key (lock) #Machine Learning (cs.LG) #Machine Learning and Algorithms #Machine learning #Mathematical optimization #Mathematics #Reinforcement Learning in Robotics #Reinforcement learning #Robot #Robotic Path Planning Algorithms #Robotics #Robotics (cs.RO) #cs.LG #cs.RO
paper · pdf · doi:10.48550/arxiv.1703.07261
published in arXiv (Cornell University) (Cornell University) · Accepted at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2017; Code at http://github.com/resibots/blackdrops; Video at http://youtu.be/kTEyYiIFGPM
openalex publication_date 2017/03/21 · arxiv created 2017/07/22 · arxiv updated 2017/07/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/08
The most data-efficient algorithms for reinforcement learning (RL) in robotics are based on uncertain dynamical models: after each episode, they first learn a dynamical model of the robot, then they use an optimization algorithm to find a policy that maximizes the expected return given the model and its uncertainties. It is often believed that this optimization can be tractable only if analytical, gradient-based algorithms are used; however, these algorithms require using specific families of reward functions and policies, which greatly limits the flexibility of the overall approach. In this paper, we introduce a novel model-based RL algorithm, called Black-DROPS (Black-box Data-efficient RObot Policy Search) that: (1) does not impose any constraint on the reward function or the policy (they are treated as black-boxes), (2) is as data-efficient as the state-of-the-art algorithm for data-efficient RL in robotics, and (3) is as fast (or faster) than analytical approaches when several cores are available. The key idea is to replace the gradient-based optimization algorithm with a parallel, black-box algorithm that takes into account the model uncertainties. We demonstrate the performance of our new algorithm on two standard control benchmark problems (in simulation) and a low-cost robotic manipulator (with a real robot).