2024/10/07 by Ehsan Futuhi, Shayan Karimi, Futuhi, Ehsan +5 · 1 citation
Engineering · Medicine · #Advanced Control Systems Optimization #Cardiovascular Function and Risk Factors #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Robotics (cs.RO)
paper · pdf · doi:10.48550/arxiv.2410.05225
openalex publication_date 2024/10/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We consider deep deterministic policy gradient (DDPG) in the context of reinforcement learning with sparse rewards. To enhance exploration, we introduce a search procedure, \emphεt-greedy, which generates exploratory options for exploring less-visited states. We prove that search using εt-greedy has polynomial sample complexity under mild MDP assumptions. To more efficiently use the information provided by rewarded transitions, we develop a new dual experience replay buffer framework, GDRB, and implement longest n-step returns. The resulting algorithm, ETGL-DDPG, integrates all three techniques: \bmεt-greedy, GDRB, and Longest n-step, into DDPG. We evaluate ETGL-DDPG on standard benchmarks and demonstrate that it outperforms DDPG, as well as other state-of-the-art methods, across all tested sparse-reward continuous environments. Ablation studies further highlight how each strategy individually enhances the performance of DDPG in this setting.