2019/08/02 by Gabrielle Kaili-May Liu, Gabrielle K. Liu, Liu, Gabrielle K. · 1 voice
Computer Science · Engineering · Mathematics · Psychology · #Algorithm #Artificial intelligence #Artificial neural network #Cognition #Cognitive psychology #Computation #Computer science #Domain Adaptation and Few-Shot Learning #Engineering #FOS: Computer and information sciences #Forgetting #Gradient descent #Human Pose and Action Recognition #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine learning #Multimodal Machine Learning Applications #Neural and Evolutionary Computing (cs.NE) #Psychology #Simple (philosophy) #Stochastic gradient descent #Task (project management) #cs.LG #cs.NE #stat.ML
paper · pdf · doi:10.48550/arxiv.1908.01052
published in arXiv (Cornell University) (Cornell University) · 9 pages, 6 figures, 1 table
openalex publication_date 2019/08/02 · arxiv published 2019/08/02 · arxiv created 2019/08/17 · arxiv updated 2019/08/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
In recent years, deep neural networks have found success in replicating human-level cognitive skills, yet they suffer from several major obstacles. One significant limitation is the inability to learn new tasks without forgetting previously learned tasks, a shortcoming known as catastrophic forgetting. In this research, we propose a simple method to overcome catastrophic forgetting and enable continual learning in neural networks. We draw inspiration from principles in neurology and physics to develop the concept of weight friction. Weight friction operates by a modification to the update rule in the gradient descent optimization method. It converges at a rate comparable to that of the stochastic gradient descent algorithm and can operate over multiple task domains. It performs comparably to current methods while offering improvements in computation and memory efficiency.