Influence-Augmented Local Simulators: A Scalable Solution for Fast Deep RL in Large Networked Systems

2022/02/03 by Miguel Suau, Jinke He, Suau, Miguel +5 · 1 voice
Computer Science · Decision Sciences · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #Simulation Techniques and Applications #cs.LG

paper · pdf · doi:10.48550/arxiv.2202.01534

openalex publication_date 2022/02/03 · arxiv published 2022/02/03 · arxiv updated 2022/02/03 · openalex created_date 2022/04/03 · openalex updated_date 2026/07/28

Abstract

Learning effective policies for real-world problems is still an open challenge for the field of reinforcement learning (RL). The main limitation being the amount of data needed and the pace at which that data can be obtained. In this paper, we study how to build lightweight simulators of complicated systems that can run sufficiently fast for deep RL to be applicable. We focus on domains where agents interact with a reduced portion of a larger environment while still being affected by the global dynamics. Our method combines the use of local simulators with learned models that mimic the influence of the global system. The experiments reveal that incorporating this idea into the deep RL workflow can considerably accelerate the training process and presents several opportunities for the future.

Discussions

• This week's #PaperILike is "Influence-Augmented Local Simulators: A Scalable Solution for Fast Deep RL in Large Networked Systems" (Suau et al., ICML 2022). Nice work on using fast local simulators to [bsky, 3 points, 0 comments]

Related

• Distributed Influence-Augmented Local Simulators for Parallel MARL in Large Networked Systems
• Sample Efficient Deep Reinforcement Learning via Local Planning
• Explicit Explore-Exploit Algorithms in Continuous State Spaces
• A Survey On Enhancing Reinforcement Learning in Complex Environments: Insights from Human and LLM Feedback
• Learning and Querying Fast Generative Models for Reinforcement Learning