2021/02/19 by Anvesh Rao Vijjini, Vijjini, Anvesh Rao, Kaveri Anuranjana +3
Computer Science · Materials Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning in Materials Science #Multimodal Machine Learning Applications #Online Learning and Analytics #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2102.09990
openalex publication_date 2021/02/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
While Curriculum Learning (CL) has recently gained traction in Natural\nlanguage Processing Tasks, it is still not adequately analyzed. Previous works\nonly show their effectiveness but fail short to explain and interpret the\ninternal workings fully. In this paper, we analyze curriculum learning in\nsentiment analysis along multiple axes. Some of these axes have been proposed\nby earlier works that need more in-depth study. Such analysis requires\nunderstanding where curriculum learning works and where it does not. Our axes\nof analysis include Task difficulty on CL, comparing CL pacing techniques, and\nqualitative analysis by visualizing the movement of attention scores in the\nmodel as curriculum phases progress. We find that curriculum learning works\nbest for difficult tasks and may even lead to a decrement in performance for\ntasks with higher performance without curriculum learning. We see that One-Pass\ncurriculum strategies suffer from catastrophic forgetting and attention\nmovement visualization within curriculum pacing. This shows that curriculum\nlearning breaks down the challenging main task into easier sub-tasks solved\nsequentially.\n