2019/03/11 by Mohammad Rostami, Rostami, Mohammad, Soheil Kolouri +3 · 5 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Topic Modeling
paper · pdf · doi:10.48550/arxiv.1903.04566
openalex publication_date 2019/03/11 · openalex created_date 2022/07/29 · openalex updated_date 2026/07/28
Despite huge success, deep networks are unable to learn effectively in\nsequential multitask learning settings as they forget the past learned tasks\nafter learning new tasks. Inspired from complementary learning systems theory,\nwe address this challenge by learning a generative model that couples the\ncurrent task to the past learned tasks through a discriminative embedding\nspace. We learn an abstract level generative distribution in the embedding that\nallows the generation of data points to represent the experience. We sample\nfrom this distribution and utilize experience replay to avoid forgetting and\nsimultaneously accumulate new knowledge to the abstract distribution in order\nto couple the current task with past experience. We demonstrate theoretically\nand empirically that our framework learns a distribution in the embedding that\nis shared across all task and as a result tackles catastrophic forgetting.\n