Formalizing the Generalization-Forgetting Trade-off in Continual Learning

2021/09/28 by Raghavan, Krishnan, Balaprakash, Prasanna · 4 citations
#Artificial Intelligence (cs.AI) #Dynamical Systems (math.DS) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Optimization and Control (math.OC)

paper · doi:10.48550/arxiv.2109.14035

Abstract

We formulate the continual learning (CL) problem via dynamic programming and model the trade-off between catastrophic forgetting and generalization as a two-player sequential game. In this approach, player 1 maximizes the cost due to lack of generalization whereas player 2 minimizes the cost due to catastrophic forgetting. We show theoretically that a balance point between the two players exists for each task and that this point is stable (once the balance is achieved, the two players stay at the balance point). Next, we introduce balanced continual learning (BCL), which is designed to attain balance between generalization and forgetting and empirically demonstrate that BCL is comparable to or better than the state of the art.

Cited by

• Erase to Retain: Low Rank Adaptation Guided Selective Unlearning in Medical Segmentation Networks
• Forgetting is Everywhere
• Ergodic Risk Measures: Towards a Risk-Aware Foundation for Continual Reinforcement Learning
• On Understanding of the Dynamics of Model Capacity in Continual Learning

Related

• The Ideal Continual Learner: An Agent That Never Forgets
• Information-Theoretic Generalization Bounds of Replay-based Continual Learning
• Theory on Forgetting and Generalization of Continual Learning
• Meta Continual Learning via Dynamic Programming
• A Comprehensive Survey of Continual Learning: Theory, Method and Application