Lifelong Learning Starting From Zero

2019/06/24 by Strannegård, Claes, Carlström, Herman, Engsner, Niklas +3
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.1906.09852

Abstract

We present a deep neural-network model for lifelong learning inspired by several forms of neuroplasticity. The neural network develops continuously in response to signals from the environment. In the beginning, the network is a blank slate with no nodes at all. It develops according to four rules: (i) expansion, which adds new nodes to memorize new input combinations; (ii) generalization, which adds new nodes that generalize from existing ones; (iii) forgetting, which removes nodes that are of relatively little use; and (iv) backpropagation, which fine-tunes the network parameters. We analyze the model from the perspective of accuracy, energy efficiency, and versatility and compare it to other network models, finding better performance in several cases.

Related

• A Deep Learning Framework for Lifelong Machine Learning
• Lifelong Neural Predictive Coding: Learning Cumulatively Online without Forgetting
• Lifelong Learning Process: Self-Memory Supervising and Dynamically Growing Networks
• Learning to Continually Learn
• On the role of neurogenesis in overcoming catastrophic forgetting