Fast deep reinforcement learning using online adjustments from the past

2018/10/18 by Steven Hansen, Pablo Sprechmann, Hansen, Steven +7 · 2 citations
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.1810.08163

Accepted at NIPS 2018

arxiv created 2018/10/18 · openalex publication_date 2018/10/18 · arxiv updated 2018/10/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose Ephemeral Value Adjusments (EVA): a means of allowing deep reinforcement learning agents to rapidly adapt to experience in their replay buffer. EVA shifts the value predicted by a neural network with an estimate of the value function found by planning over experience tuples from the replay buffer near the current state. EVA combines a number of recent ideas around combining episodic memory-like structures into reinforcement learning agents: slot-based storage, content-based retrieval, and memory-based planning. We show that EVAis performant on a demonstration task and Atari games.

Cited by

• Agentic Episodic Control
• Random Projection in Neural Episodic Control

Related

• Neural Episodic Control
• Distributed Prioritized Experience Replay
• Transferring Deep Reinforcement Learning with Adversarial Objective and Augmentation
• Sample-Efficient Deep Reinforcement Learning via Episodic Backward Update
• Enhanced Experience Replay Generation for Efficient Reinforcement Learning