Learning offline: memory replay in biological and artificial reinforcement learning

2021/09/01 by Emma L. Roscow, Raymond Chua, Rui Ponte Costa +4 · 39 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · Neuroscience · Psychology · #Artificial intelligence #Artificial neural network #Cognitive science #Computer science #Key (lock) #Memory and Neural Mechanisms #Memory consolidation #Neural dynamics and brain function #Neuroscience #Process (computing) #Psychology #Reinforcement learning #Zebrafish Biomedical Research Applications #cs.AI #cs.LG #q-bio.NC

paper · pdf · doi:10.1016/j.tins.2021.07.007

published in Trends in Neurosciences 44(10), 808-821 (Elsevier BV) · In press at Trends in Neurosciences

openalex publication_date 2021/09/01 · arxiv created 2021/09/21 · arxiv updated 2021/09/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

Learning to act in an environment to maximise rewards is among the brain's key functions. This process has often been conceptualised within the framework of reinforcement learning, which has also gained prominence in machine learning and artificial intelligence (AI) as a way to optimise decision-making. A common aspect of both biological and machine reinforcement learning is the reactivation of previously experienced episodes, referred to as replay. Replay is important for memory consolidation in biological neural networks, and is key to stabilising learning in deep neural networks. Here, we review recent developments concerning the functional roles of replay in the fields of neuroscience and AI. Complementary progress suggests how replay might support learning processes, including generalisation and continual learning, affording opportunities to transfer knowledge across the two fields to advance the understanding of biological and artificial learning and memory.

Citations

• Hippocampal Replay Is Not a Simple Function of Experience
• The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat
• Improving Generalization for Temporal Difference Learning: The Successor Representation
• Disruption of ripple‐associated hippocampal activity during rest impairs spatial learning in the rat
• Predictive Reward Signal of Dopamine Neurons
• Simple memory: a theory for archicortex
• Orbitofrontal Cortex as a Cognitive Map of Task Space
• Self-improving reactive agents based on reinforcement learning, planning and teaching
• Human-level control through deep reinforcement learning
• Mastering the game of Go with deep neural networks and tree search
• Learning without Forgetting
• Complementary learning systems within the hippocampus: a neural network modelling approach to reconciling episodic memory with statistical learning
• Hippocampal Offline Reactivation Consolidates Recently Formed Cell Assembly Patterns during Sharp Wave-Ripples
• Overcoming catastrophic forgetting in neural networks
• Continual Learning with Deep Generative Replay
• Hindsight Experience Replay
• A deep learning framework for neuroscience
• A distributional code for value in dopamine-based reinforcement learning
• Assembly-Specific Disruption of Hippocampal Replay Leads to Selective Memory Deficit
• Revisiting Fundamentals of Experience Replay
• Progressive Neural Networks