Enhanced Experience Replay Generation for Efficient Reinforcement Learning

2017/05/23 by Vincent Huang, Huang, Vincent, Tobias Ley +5 · 2 citations
Computer Science · Engineering · Neuroscience · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Neural dynamics and brain function #Reinforcement Learning in Robotics #Smart Grid Energy Management

paper · pdf · doi:10.48550/arxiv.1705.08245

openalex publication_date 2017/05/23 · openalex created_date 2017/06/05 · openalex updated_date 2026/07/28

Abstract

Applying deep reinforcement learning (RL) on real systems suffers from slow data sampling. We propose an enhanced generative adversarial network (EGAN) to initialize an RL agent in order to achieve faster learning. The EGAN utilizes the relation between states and actions to enhance the quality of data samples generated by a GAN. Pre-training the agent with the EGAN shows a steeper learning curve with a 20% improvement of training time in the beginning of learning, compared to no pre-training, and an improvement compared to training with GAN by about 5% with smaller variations. For real time systems with sparse and slow data sampling the EGAN could be used to speed up the early phases of the training process.

Citations

• Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic

Cited by

• Generative Exploration and Exploitation
• Offline-to-Online Reinforcement Learning with Classifier-Free Diffusion Generation

Related

• Synthetic Experience Replay
• Distributed Prioritized Experience Replay
• Sample Efficient Actor-Critic with Experience Replay
• Large Batch Experience Replay
• Transferring Deep Reinforcement Learning with Adversarial Objective and Augmentation