Neural Episodic Control

2017/03/06 by Alexander Pritzel, Pritzel, Alexander, Benigno Uría +14 · 2 voices · 92 citations
Computer Science · Engineering · Mathematics · Neuroscience · #Algorithm #Artificial intelligence #Artificial neural network #Bellman equation #Computer science #Control (management) #Deep learning #Deep neural networks #Engineering #Explainable Artificial Intelligence (XAI) #Function (biology) #Law #Machine learning #Mathematics #Neural dynamics and brain function #Range (aeronautics) #Reinforcement #Reinforcement Learning in Robotics #Reinforcement learning #Representation (politics) #State (computer science) #Value (mathematics) #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1703.01988

published in arXiv (Cornell University), 2827-2836 (Cornell University)

arxiv created 2017/03/06 · openalex publication_date 2017/03/06 · arxiv updated 2017/03/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Deep reinforcement learning methods attain super-human performance in a wide range of environments. Such methods are grossly inefficient, often taking orders of magnitudes more data than humans to achieve reasonable performance. We propose Neural Episodic Control: a deep reinforcement learning agent that is able to rapidly assimilate new experiences and act upon them. Our agent uses a semi-tabular representation of the value function: a buffer of past experience containing slowly changing state representations and rapidly updated estimates of the value function. We show across a wide range of environments that our agent learns significantly faster than other state-of-the-art, general purpose deep reinforcement learning agents.

Citations

• Action-Conditional Video Prediction using Deep Networks in Atari Games
• Multidimensional binary search trees used for associative searching
• Prioritized Experience Replay
• Deep Exploration via Bootstrapped DQN
• Strategic Attentive Writer for Learning Macro-Actions
• PathNet: Evolution Channels Gradient Descent in Super Neural Networks

Cited by

• Short-Term-to-Long-Term Memory Transfer for Knowledge Graphs under Partial Observability
• Coordinating from Memory: Graph-Structured Experience Reuse for Multi-Agent Adaptation in Dynamic Manufacturing
• Neuro-Symbolic Meta-Policies for Temporal Knowledge-Graph Memory under Partial Observability
• Latent learning: episodic memory complements parametric learning by enabling flexible reuse of experiences
• RANa: Retrieval-Augmented Navigation
• Beyond Heuristics: A Decision-Theoretic Framework for Agent Memory Management
• MemVerse: Multimodal Memory for Lifelong Learning Agents
• Improving Human-Robot Teamwork in Urban Search and Rescue Through Episodic Memory of Prior Collaboration
• Energy-Efficient Domain-Specific Artificial Intelligence Models and Agents: Pathways and Paradigms
• Dejavu: Towards Experience Feedback Learning for Embodied Intelligence
• Multigrid Neural Memory
• Model-Based Episodic Memory Induces Dynamic Hybrid Controls
• Training Agents using Upside-Down Reinforcement Learning
• Memory-Driven Self-Improvement for Decision Making with Large Language Models
• Embodied AI: From LLMs to World Models
• Memory-Augmented Potential Field Theory: A Framework for Adaptive Control in Non-Convex Domains
• Policy Compatible Skill Incremental Learning via Lazy Learning Interface
• Mental Accounts for Actions: EWA-Inspired Attention in Decision Transformers
• WebSight: A Vision-First Architecture for Robust Web Agents
• Memento: Fine-tuning LLM Agents without Fine-tuning LLMs
• CNN with large memory layers
• Generalizable Episodic Memory for Deep Reinforcement Learning
• A Short Survey On Memory Based Reinforcement Learning
• Replay in Deep Learning: Current Approaches and Missing Biological Elements
• The Benchmark Lottery
• Learning to Learn Variational Semantic Memory
• Deep Graph Memory Networks for Forgetting-Robust Knowledge Tracing
• State-based Episodic Memory for Multi-Agent Reinforcement Learning
• Persistent Message Passing
• TriVLA: A Triple-System-Based Unified Vision-Language-Action Model with Episodic World Modeling for General Robot Control
• Sample-Efficient Reinforcement Learning with Maximum Entropy Mellowmax Episodic Control
• Memory-Efficient Episodic Control Reinforcement Learning with Dynamic Online k-means
• A Proposal for Intelligent Agents with Episodic Memory
• Better transfer learning with inferred successor maps
• RAN Cortex: Memory-Augmented Intelligence for Context-Aware Decision-Making in AI-Native Networks
• Agentic Episodic Control
• A Memory-Augmented Neural Network Model of Abstract Rule Learning
• Improving Sample Efficiency with Normalized RBF Kernels
• Graph Element Networks: adaptive, structured computation and memory
• Biologically inspired architectures for sample-efficient deep reinforcement learning
• Random Projection in Neural Episodic Control
• Asynchronous Episodic Deep Deterministic Policy Gradient: Towards Continuous Control in Computationally Complex Environments
• Self-Imitation Learning for Robot Tasks with Sparse and Delayed Rewards
• Emotions in Artificial Intelligence
• Selective Memory for Artificial Intelligence: Write-Time Gating with Hierarchical Archiving
• Multitask Adaptation by Retrospective Exploration with Learned World Models
• Coupled Control, Structured Memory, and Verifiable Action in Agentic AI (SCRAT -- Stochastic Control with Retrieval and Auditable Trajectories): A Comparative Perspective from Squirrel Locomotion and Scatter-Hoarding
• MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory
• Caching for the Future: Scrub Jay Episodic Memory Principles for Agent Memory Systems
• Continual and Multi-task Reinforcement Learning With Shared Episodic Memory
• Multimodal Perception for Goal-oriented Navigation: A Survey
• Learning Guidance Rewards with Trajectory-space Smoothing
• Memory-Based Graph Networks
• Siamese Network with Dual Attention for EEG-Driven Social Learning: Bridging the Human-Robot Gap in Long-Tail Autonomous Driving

Discussions

• Neural Episodic Control [hn, 50 points, 5 comments]
• Neural Episodic Control – new breaktrough in deep reinforcement learning [hn, 4 points, 0 comments]

Related

• Model-Free Episodic Control
• Successor Feature Neural Episodic Control
• Memory-Efficient Episodic Control Reinforcement Learning with Dynamic Online k-means
• Neural Episodic Control with State Abstraction
• Random Projection in Neural Episodic Control