Revisiting Rainbow: Promoting more Insightful and Inclusive Deep\n Reinforcement Learning Research

2020/11/20 by Johan Obando-Ceron, Pablo Samuel Castro, Obando-Ceron, Johan S. +1 · 4 citations
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Artificial Intelligence (cs.AI) #Data Stream Mining Techniques #Evolutionary Algorithms and Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2011.14826

openalex publication_date 2020/11/20 · openalex created_date 2022/10/02 · openalex updated_date 2026/07/28

Abstract

Since the introduction of DQN, a vast majority of reinforcement learning\nresearch has focused on reinforcement learning with deep neural networks as\nfunction approximators. New methods are typically evaluated on a set of\nenvironments that have now become standard, such as Atari 2600 games. While\nthese benchmarks help standardize evaluation, their computational cost has the\nunfortunate side effect of widening the gap between those with ample access to\ncomputational resources, and those without. In this work we argue that, despite\nthe community's emphasis on large-scale environments, the traditional\nsmall-scale environments can still yield valuable scientific insights and can\nhelp reduce the barriers to entry for underprivileged communities. To\nsubstantiate our claims, we empirically revisit the paper which introduced the\nRainbow algorithm [Hessel et al., 2018] and present some new insights into the\nalgorithms used by Rainbow.\n

Citations

• Robust Estimation of a Location Parameter
• Dueling Network Architectures for Deep Reinforcement Learning
• How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experiments

Cited by

• Recurrent Off-Policy Deep Reinforcement Learning Doesn't Have to be Slow
• Representation Learning Enables Scalable Multitask Deep Reinforcement Learning
• The Formalism-Implementation Gap in Reinforcement Learning Research
• Octax: Accelerated CHIP-8 Arcade Environments for Reinforcement Learning in JAX

Related

• Rainbow: Combining Improvements in Deep Reinforcement Learning
• Beyond The Rainbow: High Performance Deep Reinforcement Learning on a Desktop PC
• Is Deep Reinforcement Learning Really Superhuman on Atari? Leveling the playing field
• Deep Reinforcement Learning: An Overview
• Implementing the Deep Q-Network