2021/03/29 by Andrea Rosasco, Antonio Carta, Rosasco, Andrea +7
Computer Science · #Artificial Intelligence (cs.AI) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #cs.AI #cs.LG
paper · pdf · doi:10.48550/arxiv.2103.15851
openalex publication_date 2021/03/29 · arxiv created 2021/06/22 · arxiv updated 2021/06/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Replay strategies are Continual Learning techniques which mitigate catastrophic forgetting by keeping a buffer of patterns from previous experiences, which are interleaved with new data during training. The amount of patterns stored in the buffer is a critical parameter which largely influences the final performance and the memory footprint of the approach. This work introduces Distilled Replay, a novel replay strategy for Continual Learning which is able to mitigate forgetting by keeping a very small buffer (1 pattern per class) of highly informative samples. Distilled Replay builds the buffer through a distillation process which compresses a large dataset into a tiny set of informative examples. We show the effectiveness of our Distilled Replay against popular replay-based strategies on four Continual Learning benchmarks.