On Tiny Episodic Memories in Continual Learning

2019/02/27 by Arslan Chaudhry, Marcus Rohrbach, Chaudhry, Arslan +11 · 23 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning and ELM

paper · pdf · doi:10.48550/arxiv.1902.10486

openalex publication_date 2019/02/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In continual learning (CL), an agent learns from a stream of tasks leveraging prior experience to transfer knowledge to future tasks. It is an ideal framework to decrease the amount of supervision in the existing learning algorithms. But for a successful knowledge transfer, the learner needs to remember how to perform previous tasks. One way to endow the learner the ability to perform tasks seen in the past is to store a small memory, dubbed episodic memory, that stores few examples from previous tasks and then to replay these examples when training for future tasks. In this work, we empirically analyze the effectiveness of a very small episodic memory in a CL setup where each training example is only seen once. Surprisingly, across four rather different supervised learning benchmarks adapted to CL, a very simple baseline, that jointly trains on both examples from the current task as well as examples stored in the episodic memory, significantly outperforms specifically designed CL approaches with and without episodic memory. Interestingly, we find that repetitive training on even tiny memories of past tasks does not harm generalization, on the contrary, it improves it, with gains between 7% and 17% when the memory is populated with a single example per class.

Citations

• Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning
• Overcoming Catastrophic Forgetting by Incremental Moment Matching
• Experience Replay for Continual Learning
• Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Sophia: A Persistent Agent Framework of Artificial Life
• Sequencing to Mitigate Catastrophic Forgetting in Continual Learning
• Harmonious Parameter Adaptation in Continual Visual Instruction Tuning for Safety-Aligned MLLMs
• PANDA -- Patch And Distribution-Aware Augmentation for Long-Tailed Exemplar-Free Continual Learning
• Benchmarking Catastrophic Forgetting Mitigation Methods in Federated Time Series Forecasting
• Learning After Model Deployment
• DOLFIN: Balancing Stability and Plasticity in Federated Continual Learning
• DATS: Distance-Aware Temperature Scaling for Calibrated Class-Incremental Learning
• DECODE: Tackling Representation and Decision Degradation in Continual AI-Generated Image Detection
• Linear Mode Connectivity in Multitask and Continual Learning
• Online Coreset Selection for Rehearsal-based Continual Learning
• The Effectiveness of Memory Replay in Large Scale Continual Learning
• Meta-Learning Representations for Continual Learning
• Wide Neural Networks Forget Less Catastrophically
• Lifelong Learner: Discovering Versatile Neural Solvers for Vehicle Routing Problems
• LoRA in LoRA: Towards Parameter-Efficient Architecture Expansion for Continual Visual Instruction Tuning
• Prediction-Oriented Subsampling from Data Streams
• Routing Networks with Co-training for Continual Learning
• Batch-level Experience Replay with Review for Continual Learning
• Layerwise Optimization by Gradient Decomposition for Continual Learning
• Graph-Based Continual Learning
• An EM Framework for Online Incremental Learning of Semantic Segmentation
• iLGaCo: Incremental Learning of Gait Covariate Factors

Related

• Gradient Episodic Memory for Continual Learning
• Using Hindsight to Anchor Past Knowledge in Continual Learning
• Saliency-Augmented Memory Completion for Continual Learning
• Carousel Memory: Rethinking the Design of Episodic Memory for Continual Learning
• Insights from the Future for Continual Learning