Automated Reinforcement Learning: An Overview

2022/01/13 by Reza Refaei Afshar, Afshar, Reza Refaei, Joaquin Vanschoren +11 · 13 citations
Business, Management and Accounting · Computer Science · Engineering · Mathematics · #Action selection #Artificial intelligence #Computer science #Data Stream Mining Techniques #Engineering #Machine learning #Markov decision process #Markov process #Mathematics #Reinforcement Learning in Robotics #Reinforcement learning #Selection (genetic algorithm) #State space #Supply Chain and Inventory Management #Task (project management) #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2201.05000

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2022/01/13 · openalex created_date 2022/05/05 · openalex updated_date 2026/07/28

Abstract

Reinforcement Learning and, recently, Deep Reinforcement Learning are popular methods for solving sequential decision-making problems modeled as Markov Decision Processes. RL modeling of a problem and selecting algorithms and hyper-parameters require careful consideration, as different configurations may entail completely different performances. These considerations are mainly the task of RL experts; however, RL is progressively becoming popular in other fields, such as combinatorial optimization, where researchers and system designers are not necessarily RL experts. Besides, many modeling decisions are typically made manually, such as defining state and action space, size of batches, batch update frequency, and time steps. For these reasons, automating different components of RL is of great importance, and it has attracted much attention in recent years. Automated RL provides a framework in which different components of RL, including MDP modeling, algorithm selection, and hyper-parameter optimization, are modeled and defined automatically. In this article, we present the literature on automated RL (AutoRL), including the recent large language model (LLM) based techniques. We also discuss the recent work on techniques that are not presently tailored for automated RL but hold promise for future integration into AutoRL. Furthermore, we discuss the challenges, open questions, and research directions in AutoRL.

Citations

• Large Language Models as Optimizers
• Deep Reinforcement Learning that Matters
• Learning Combinatorial Optimization Algorithms over Graphs
• Evolving Deep Neural Networks
• Learning to reinforcement learn
• Neural Architecture Search with Reinforcement Learning
• Learning to learn by gradient descent by gradient descent
• Gradient-based Hyperparameter Optimization through Reversible Learning
• DeepWalk

Related

• Automated Reinforcement Learning (AutoRL): A Survey and Open Problems
• ARLO: A Framework for Automated Reinforcement Learning
• Statistical and Algorithmic Foundations of Reinforcement Learning
• Reinforcement Learning: An Overview
• Sample-Efficient Automated Deep Reinforcement Learning