A Survey on Reproducibility by Evaluating Deep Reinforcement Learning Algorithms on Real-World Robots

2019/09/09 by Nicolai A. Lynnerup, Lynnerup, Nicolai A., Laura Nolling +5 · 3 citations
Computer Science · Mathematics · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #Mobile Crowdsensing and Crowdsourcing #Reinforcement Learning in Robotics #Robotics (cs.RO) #cs.AI #cs.LG #cs.RO #stat.ML

paper · pdf · doi:10.48550/arxiv.1909.03772

Appears in Proceedings of the Third Conference on Robot Learning (CoRL 2019). Companion source code at https://github.com/dti-research/SenseActExperiments/

openalex publication_date 2019/09/09 · arxiv created 2019/09/11 · arxiv updated 2019/09/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

As reinforcement learning (RL) achieves more success in solving complex tasks, more care is needed to ensure that RL research is reproducible and that algorithms herein can be compared easily and fairly with minimal bias. RL results are, however, notoriously hard to reproduce due to the algorithms' intrinsic variance, the environments' stochasticity, and numerous (potentially unreported) hyper-parameters. In this work we investigate the many issues leading to irreproducible research and how to manage those. We further show how to utilise a rigorous and standardised evaluation approach for easing the process of documentation, evaluation and fair comparison of different algorithms, where we emphasise the importance of choosing the right measurement metrics and conducting proper statistics on the results, for unbiased reporting of the results.

Citations

• Trust Region Policy Optimization
• Electronic documents give reproducible research a new meaning
• Continuous control with deep reinforcement learning
• Prioritized Experience Replay
• Reinforcement Learning with Deep Energy-Based Policies
• Reproducibility vs. Replicability: A Brief History of a Confused Terminology
• A Study on Overfitting in Deep Reinforcement Learning
• Deep Reinforcement Learning that Matters

Cited by

• Predictive Analytics for Collaborators Answers, Code Quality, and Dropout on Stack Overflow
• Behavior-Consistent Deep Reinforcement Learning
• AlgOS: Algorithm Operating System

Related

• Deep Reinforcement Learning that Matters
• Evaluating the Performance of Reinforcement Learning Algorithms
• Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes
• Deep Reinforcement Learning at the Edge of the Statistical Precipice
• Beyond Expected Return: Accounting for Policy Reproducibility when Evaluating Reinforcement Learning Algorithms