Overcoming classic challenges for artificial neural networks by providing incentives and practice

2024/10/14 by Kazuki Irie, Brenden M. Lake, Irie, Kazuki +1 · 3 voices · 3 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · Psychology · #Artificial intelligence #Artificial neural network #Business #Child and Animal Learning Development #Computer science #Data science #Domain Adaptation and Few-Shot Learning #Neural Networks and Applications #cs.AI #cs.LG #q-bio.NC

paper · pdf · doi:10.48550/arxiv.2410.10596

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2024/10/14 · openalex created_date 2024/10/20 · openalex updated_date 2026/07/28

Abstract

Since the earliest proposals for artificial neural network (ANN) models of the mind and brain, critics have pointed out key weaknesses in these models compared to human cognitive abilities. Here we review recent work that uses metalearning to overcome several classic challenges, which we characterize as addressing the Problem of Incentive and Practice -- that is, providing machines with both incentives to improve specific skills and opportunities to practice those skills. This explicit optimization contrasts with more conventional approaches that hope the desired behaviour will emerge through optimizing related but different objectives. We review applications of this principle to addressing four classic challenges for ANNs: systematic generalization, catastrophic forgetting, few-shot learning and multi-step reasoning. We also discuss how large language models incorporate key aspects of this metalearning framework (namely, sequence prediction with feedback trained on diverse data), which helps to explain some of their successes on these classic challenges. Finally, we discuss the prospects for understanding aspects of human development through this framework, and whether natural environments provide the right incentives and practice for learning how to make challenging generalizations.

Cited by

• Fast weight programming and linear transformers: from machine learning to neurobiology
• Whither symbols in the era of advanced neural networks?
• Teasing Apart Architecture and Initial Weights as Sources of Inductive Bias in Neural Networks

Discussions

• A lovely overview of meta-learning, or as they call it, networks that improve through practice arxiv.org/abs/2410.10596 Found by a PhD student in our group [bsky, 72 points, 0 comments]
• This updated paper on improved meta-learning for training AI models struck me as an opportunity also for ICL, so I updated my Claude.ai Project Instructions Engineering meta-prompt to account for the [bsky, 0 points, 1 comments]
• 4/ Kazuki Irie shares work with Brenden Lake on meta-learning advanced behaviors like continual, compositional, and few-shot learning end-to-end - behaviors they argue cannot be hand-engineered. https [bsky, 0 points, 1 comments]

Related

• One Big Net For Everything
• Towards Brain Inspired Design for Addressing the Shortcomings of ANNs
• Human-like Cognitive Generalization for Large Models via Brain-in-the-loop Supervision
• Towards continual task learning in artificial neural networks: current approaches and insights from neuroscience
• It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimization