2021/10/20 by Soumya Banerjee, Vinay Kumar Verma, Banerjee, Soumya +7
Computer Science · Medicine · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Data Stream Mining Techniques #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neonatal and fetal brain pathology
paper · pdf · doi:10.48550/arxiv.2110.10741
openalex publication_date 2021/10/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
A wide variety of methods have been developed to enable lifelong learning in conventional deep neural networks. However, to succeed, these methods require a `batch' of samples to be available and visited multiple times during training. While this works well in a static setting, these methods continue to suffer in a more realistic situation where data arrives in online streaming manner. We empirically demonstrate that the performance of current approaches degrades if the input is obtained as a stream of data with the following restrictions: (i) each instance comes one at a time and can be seen only once, and (ii) the input data violates the i.i.d assumption, i.e., there can be a class-based correlation. We propose a novel approach (CIOSL) for the class-incremental learning in an online streaming setting to address these challenges. The proposed approach leverages implicit and explicit dual weight regularization and experience replay. The implicit regularization is leveraged via the knowledge distillation, while the explicit regularization incorporates a novel approach for parameter regularization by learning the joint distribution of the buffer replay and the current sample. Also, we propose an efficient online memory replay and replacement buffer strategy that significantly boosts the model's performance. Extensive experiments and ablation on challenging datasets show the efficacy of the proposed method.