Maximum Entropy Hindsight Experience Replay

2024/10/31 by Crowder, Douglas C., Trappett, Matthew L., McKenzie, Darrien M. +1
#FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2410.24016

Abstract

Hindsight experience replay (HER) is well-known to accelerate goal-based reinforcement learning (RL). While HER is generally applied to off-policy RL algorithms, we previously showed that HER can also accelerate on-policy algorithms, such as proximal policy optimization (PPO), for goal-based Predator-Prey environments. Here, we show that we can improve the previous PPO-HER algorithm by selectively applying HER in a principled manner.

Related

• Hindsight Experience Replay Accelerates Proximal Policy Optimization
• Bias-Reduced Hindsight Experience Replay with Virtual Goal Prioritization
• MHER: Model-based Hindsight Experience Replay
• Learning and reusing primitive behaviours to improve Hindsight Experience Replay sample efficiency
• USHER: Unbiased Sampling for Hindsight Experience Replay