Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions.

1990/04/01 by Roger Ratcliff · 69 citations
Neuroscience · Computer Science · #Memory Processes and Influences #Domain Adaptation and Few-Shot Learning #Topic Modeling

paper · doi:10.1037/0033-295x.97.2.285

Abstract

Multilayer connectionist models of memory based on the encoder model using the backpropagation learning rule are evaluated. The models are applied to standard recognition memory procedures in which items are studied sequentially and then tested for retention. Sequential learning in these models leads to 2 major problems. First, well-learned information is forgotten rapidly as new information is learned. Second, discrimination between studied items and new items either decreases or is nonmonotonic as a function of learning. To address these problems, manipulations of the network within the multilayer model and several variants of the multilayer model were examined, including a model with prelearned memory and a context model, but none solved the problems. The problems discussed provide limitations on connectionist models applied to human memory and in tasks where information to be learned is not all available during learning.

Cited by

• Why the Brain Consolidates: Predictive Forgetting for Optimal Generalisation
• Shared sensitivity to data distribution during learning in humans and transformer networks
• The Cake that is Intelligence and Who Gets to Bake it: An AI Analogy and its Implications for Participation
• Algorithmic insights on continual learning from fruit flies
• Towards Neural Network Patching: Evaluating Engagement-Layers and Patch-Architectures
• Towards continual learning in medical imaging
• Variational Prototype Replays for Continual Learning
• On the Opportunities and Risks of Foundation Models
• Learning continually with representational drift
• Provable Lifelong Learning of Representations
• Efficient Continual Learning in Neural Machine Translation: A Low-Rank Adaptation Approach
• Superposition of many models into one
• DeepObliviate: A Powerful Charm for Erasing Data Residual Memory in Deep Neural Networks
• Learning Sparse Representations Incrementally in Deep Reinforcement Learning
• SuRe: Surprise-Driven Prioritised Replay for Continual LLM Learning
• Continual Learning with Deep Generative Replay
• Rethinking Experience Replay: a Bag of Tricks for Continual Learning
• ConSurv: Multimodal Continual Learning for Survival Analysis
• Improving Sustainability of Adversarial Examples in Class-Incremental Learning
• Learning with Preserving for Continual Multitask Learning
• RMM: Reinforced Memory Management for Class-Incremental Learning
• The brain as a blueprint: a survey of brain-inspired approaches to learning in artificial intelligence
• Fast Model Editing at Scale
• Deep neural networks and humans both benefit from compositional language structure
• Catastrophic Forgetting, Rehearsal and Pseudorehearsal
• S2OSC: A Holistic Semi-Supervised Approach for Open Set Classification
• Pseudo-Rehearsal for Continual Learning with Normalizing Flows
• End-to-End Incremental Learning
• Locally Weighted Regression Pseudo-Rehearsal for Online Learning of Vehicle Dynamics
• Orthogonal Gradient Descent for Continual Learning
• Education Paradigm Shift To Maintain Human Competitive Advantage Over AI
• Continual Knowledge Consolidation LORA for Domain Incremental Learning
• COLA: Continual Learning via Autoencoder Retrieval of Adapters
• KORE: Enhancing Knowledge Injection for Large Multimodal Models via Knowledge-Oriented Controls
• Mapping Post-Training Forgetting in Language Models at Scale
• Evaluating Online Continual Learning with CALM
• Kronecker Factorization for Preventing Catastrophic Forgetting in Large-scale Medical Entity Linking
• Barriers for Learning in an Evolving World: Mathematical Understanding of Loss of Plasticity
• LEAF: A Robust Expert-Based Framework for Few-Shot Continual Event Detection
• Continual Learning to Generalize Forwarding Strategies for Diverse Mobile Wireless Networks
• DATS: Distance-Aware Temperature Scaling for Calibrated Class-Incremental Learning
• Does the Adam Optimizer Exacerbate Catastrophic Forgetting?
• Assemblies, synapse clustering and network topology interact with plasticity to explain structure-function relationships of the cortical connectome
• Key-value memory in the brain
• Security and Privacy Challenges of Large Language Models: A Survey
• Carousel Memory: Rethinking the Design of Episodic Memory for Continual Learning
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• The Natural Language Decathlon: Multitask Learning as Question Answering
• Bilevel Continual Learning
• Positive-Congruent Training: Towards Regression-Free Model Updates
• Extracting Training Data from Large Language Models
• Incremental Learning with Maximum Entropy Regularization: Rethinking\n Forgetting and Intransigence
• Pseudorehearsal in actor-critic agents
• Editable Neural Networks
• Can Synthetic Faces Undo the Damage of Dataset Bias to Face Recognition and Facial Landmark Detection?
• aims-PAX: Parallel Active eXploration for the automated construction of Machine Learning Force Fields
• Adaptive Variance-Penalized Continual Learning with Fisher Regularization
• Memorisation and forgetting in a learning Hopfield neural network: bifurcation mechanisms, attractors and basins
• Continual Learning in Neural Networks
• ETA: Energy-based Test-time Adaptation for Depth Completion
• Extending Foundational Monocular Depth Estimators to Fisheye Cameras with Calibration Tokens
• Prediction-Oriented Subsampling from Data Streams
• Replay in Deep Learning: Current Approaches and Missing Biological Elements
• Boosting Soft Actor-Critic: Emphasizing Recent Experience without Forgetting the Past
• Striving for Simplicity and Performance in Off-Policy DRL: Output Normalization and Non-Uniform Sampling
• Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics
• A Unifying Bayesian View of Continual Learning
• Graph-Based Continual Learning
• Generative replay with feedback connections as a general strategy for continual learning

Related

• A model of memory, learning and recognition
• A Neural Network Model of Complementary Learning Systems: Pattern Separation and Completion for Continual Learning
• Memory Networks: Towards Fully Biologically Plausible Learning
• Supervised Generative Reconstruction: An Efficient Way To Flexibly Store and Recognize Patterns
• On the proper treatment of connectionism