Neural Episodic Control

2017/03/06 by Alexander Pritzel, Pritzel, Alexander, Benigno Uría +14 · 2 voices · 24 citations
Computer Science · Neuroscience · #Explainable Artificial Intelligence (XAI) #Neural dynamics and brain function #Reinforcement Learning in Robotics #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1703.01988

openalex publication_date 2017/03/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Deep reinforcement learning methods attain super-human performance in a wide range of environments. Such methods are grossly inefficient, often taking orders of magnitudes more data than humans to achieve reasonable performance. We propose Neural Episodic Control: a deep reinforcement learning agent that is able to rapidly assimilate new experiences and act upon them. Our agent uses a semi-tabular representation of the value function: a buffer of past experience containing slowly changing state representations and rapidly updated estimates of the value function. We show across a wide range of environments that our agent learns significantly faster than other state-of-the-art, general purpose deep reinforcement learning agents.

Citations

• Action-Conditional Video Prediction using Deep Networks in Atari Games
• Multidimensional binary search trees used for associative searching
• Prioritized Experience Replay
• Deep Exploration via Bootstrapped DQN
• Strategic Attentive Writer for Learning Macro-Actions
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks

Cited by

• Short-Term-to-Long-Term Memory Transfer for Knowledge Graphs under Partial Observability
• Coordinating from Memory: Graph-Structured Experience Reuse for Multi-Agent Adaptation in Dynamic Manufacturing
• Neuro-Symbolic Meta-Policies for Temporal Knowledge-Graph Memory under Partial Observability
• Latent learning: episodic memory complements parametric learning by enabling flexible reuse of experiences
• RANa: Retrieval-Augmented Navigation
• Beyond Heuristics: A Decision-Theoretic Framework for Agent Memory Management
• MemVerse: Multimodal Memory for Lifelong Learning Agents
• Improving Human-Robot Teamwork in Urban Search and Rescue Through Episodic Memory of Prior Collaboration
• Energy-Efficient Domain-Specific Artificial Intelligence Models and Agents: Pathways and Paradigms
• Dejavu: Towards Experience Feedback Learning for Embodied Intelligence
• Multigrid Neural Memory
• Model-Based Episodic Memory Induces Dynamic Hybrid Controls
• Training Agents using Upside-Down Reinforcement Learning
• Memory-Driven Self-Improvement for Decision Making with Large Language Models
• Embodied AI: From LLMs to World Models
• Memory-Augmented Potential Field Theory: A Framework for Adaptive Control in Non-Convex Domains
• Policy Compatible Skill Incremental Learning via Lazy Learning Interface
• Mental Accounts for Actions: EWA-Inspired Attention in Decision Transformers
• WebSight: A Vision-First Architecture for Robust Web Agents
• Memento: Fine-tuning LLM Agents without Fine-tuning LLMs
• CNN with large memory layers
• Generalizable Episodic Memory for Deep Reinforcement Learning
• A Short Survey On Memory Based Reinforcement Learning
• Replay in Deep Learning: Current Approaches and Missing Biological Elements

Discussions

• Neural Episodic Control [hn, 50 points, 5 comments]
• Neural Episodic Control – new breaktrough in deep reinforcement learning [hn, 4 points, 0 comments]

Related

• Model-Free Episodic Control
• Successor Feature Neural Episodic Control
• Memory-Efficient Episodic Control Reinforcement Learning with Dynamic Online k-means
• Neural Episodic Control with State Abstraction
• Random Projection in Neural Episodic Control