Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions.

1990/04/01 by Roger Ratcliff · 881 citations
Computer Science · Neuroscience · Psychology · #Artificial intelligence #Artificial neural network #Backpropagation #Cognitive psychology #Cognitive science #Computer science #Connectionism #Context (archaeology) #Domain Adaptation and Few-Shot Learning #Forgetting #Machine learning #Memory Processes and Influences #Memory model #Psychology #Topic Modeling

paper · doi:10.1037/0033-295x.97.2.285

published in Psychological Review 97(2), 285-308 (American Psychological Association)

openalex publication_date 1990/04/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/04

Abstract

Multilayer connectionist models of memory based on the encoder model using the backpropagation learning rule are evaluated. The models are applied to standard recognition memory procedures in which items are studied sequentially and then tested for retention. Sequential learning in these models leads to 2 major problems. First, well-learned information is forgotten rapidly as new information is learned. Second, discrimination between studied items and new items either decreases or is nonmonotonic as a function of learning. To address these problems, manipulations of the network within the multilayer model and several variants of the multilayer model were examined, including a model with prelearned memory and a context model, but none solved the problems. The problems discussed provide limitations on connectionist models applied to human memory and in tasks where information to be learned is not all available during learning.

Cited by

• Why the Brain Consolidates: Predictive Forgetting for Optimal Generalisation
• Shared sensitivity to data distribution during learning in humans and transformer networks
• The Cake that is Intelligence and Who Gets to Bake it: An AI Analogy and its Implications for Participation
• Algorithmic insights on continual learning from fruit flies
• Towards Neural Network Patching: Evaluating Engagement-Layers and Patch-Architectures
• Towards continual learning in medical imaging
• Variational Prototype Replays for Continual Learning
• On the Opportunities and Risks of Foundation Models
• Learning continually with representational drift
• Provable Lifelong Learning of Representations
• Efficient Continual Learning in Neural Machine Translation: A Low-Rank Adaptation Approach
• Superposition of many models into one
• DeepObliviate: A Powerful Charm for Erasing Data Residual Memory in Deep Neural Networks
• Learning Sparse Representations Incrementally in Deep Reinforcement Learning
• SuRe: Surprise-Driven Prioritised Replay for Continual LLM Learning
• Continual Learning with Deep Generative Replay
• Rethinking Experience Replay: a Bag of Tricks for Continual Learning
• ConSurv: Multimodal Continual Learning for Survival Analysis
• Improving Sustainability of Adversarial Examples in Class-Incremental Learning
• Learning with Preserving for Continual Multitask Learning
• RMM: Reinforced Memory Management for Class-Incremental Learning
• The brain as a blueprint: a survey of brain-inspired approaches to learning in artificial intelligence
• Fast Model Editing at Scale
• Deep neural networks and humans both benefit from compositional language structure
• Catastrophic Forgetting, Rehearsal and Pseudorehearsal
• S2OSC: A Holistic Semi-Supervised Approach for Open Set Classification
• Pseudo-Rehearsal for Continual Learning with Normalizing Flows
• End-to-End Incremental Learning
• Locally Weighted Regression Pseudo-Rehearsal for Online Learning of Vehicle Dynamics
• Orthogonal Gradient Descent for Continual Learning
• Education Paradigm Shift To Maintain Human Competitive Advantage Over AI
• Continual Knowledge Consolidation LORA for Domain Incremental Learning
• COLA: Continual Learning via Autoencoder Retrieval of Adapters
• KORE: Enhancing Knowledge Injection for Large Multimodal Models via Knowledge-Oriented Controls
• Mapping Post-Training Forgetting in Language Models at Scale
• Evaluating Online Continual Learning with CALM
• Kronecker Factorization for Preventing Catastrophic Forgetting in Large-scale Medical Entity Linking
• Barriers for Learning in an Evolving World: Mathematical Understanding of Loss of Plasticity
• LEAF: A Robust Expert-Based Framework for Few-Shot Continual Event Detection
• Continual Learning to Generalize Forwarding Strategies for Diverse Mobile Wireless Networks
• DATS: Distance-Aware Temperature Scaling for Calibrated Class-Incremental Learning
• Does the Adam Optimizer Exacerbate Catastrophic Forgetting?
• Assemblies, synapse clustering and network topology interact with plasticity to explain structure-function relationships of the cortical connectome
• Key-value memory in the brain
• Security and Privacy Challenges of Large Language Models: A Survey
• Carousel Memory: Rethinking the Design of Episodic Memory for Continual Learning
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
• The Natural Language Decathlon: Multitask Learning as Question Answering
• Bilevel Continual Learning
• Positive-Congruent Training: Towards Regression-Free Model Updates
• Extracting Training Data from Large Language Models
• Incremental Learning with Maximum Entropy Regularization: Rethinking Forgetting and Intransigence
• Pseudorehearsal in actor-critic agents
• Editable Neural Networks
• Can Synthetic Faces Undo the Damage of Dataset Bias to Face Recognition and Facial Landmark Detection?
• aims-PAX: Parallel Active eXploration for the automated construction of Machine Learning Force Fields
• Adaptive Variance-Penalized Continual Learning with Fisher Regularization
• Composed Fine-Tuning: Freezing Pre-Trained Denoising Autoencoders for\n Improved Generalization
• Memorisation and forgetting in a learning Hopfield neural network: bifurcation mechanisms, attractors and basins
• Continual Learning in Neural Networks
• ETA: Energy-based Test-time Adaptation for Depth Completion
• Extending Foundational Monocular Depth Estimators to Fisheye Cameras with Calibration Tokens
• Prediction-Oriented Subsampling from Data Streams
• Replay in Deep Learning: Current Approaches and Missing Biological Elements
• Boosting Soft Actor-Critic: Emphasizing Recent Experience without Forgetting the Past
• Striving for Simplicity and Performance in Off-Policy DRL: Output Normalization and Non-Uniform Sampling
• Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics
• Word Frequency Does Not Predict Grammatical Knowledge in Language Models
• A Unifying Bayesian View of Continual Learning
• Overcoming Catastrophic Interference by Conceptors
• CCL-XCoT: An Efficient Cross-Lingual Knowledge Transfer Method for Mitigating Hallucination Generation
• Graph-Based Continual Learning
• Generative replay with feedback connections as a general strategy for continual learning
• Foundation Models as Class-Incremental Learners for Dermatological Image Classification
• Continual Learning with Adaptive Weights (CLAW)
• Quasiregularity and Its Discontents: The Legacy of the Past Tense Debate
• Lifelong Learning of Hate Speech Classification on Social Media
• Gradient Episodic Memory with a Soft Constraint for Continual Learning
• Catastrophic Forgetting Mitigation Through Plateau Phase Activity Profiling
• Online Reinforcement Learning Control by Direct Heuristic Dynamic Programming: from Time-Driven to Event-Driven
• The Bayesian Approach to Continual Learning: An Overview
• Why Reinforcement Fine-Tuning Enables MLLMs Preserve Prior Knowledge Better: A Data Perspective
• Variational Auto-Regressive Gaussian Processes for Continual Learning
• Toward Understanding Catastrophic Forgetting in Continual Learning
• Continual Learning with Columnar Spiking Neural Networks
• SEAT: Sparse Entity-Aware Tuning for Knowledge Adaptation while Preserving Epistemic Abstention
• Training Networks in Null Space of Feature Covariance for Continual Learning
• Lifelong Object Detection
• Solving hybrid machine learning tasks by traversing weight space geodesics
• Lyapunov Learning at the Onset of Chaos
• Overcoming Catastrophic Forgetting by Generative Regularization
• EKPC: Elastic Knowledge Preservation and Compensation for Class-Incremental Learning
• Class-Incremental Learning for Honey Botanical Origin Classification with Hyperspectral Images: A Study with Continual Backpropagation
• Jointly Learning from Decentralized (Federated) and Centralized Data to Mitigate Distribution Shift
• Hierarchical-Task-Aware Multi-modal Mixture of Incremental LoRA Experts for Embodied Continual Learning
• A Review of Computer Vision Methods in Network Security
• Localizing Catastrophic Forgetting in Neural Networks
• DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations
• Class Incremental Learning for Algorithm Selection
• L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning
• Center Loss Regularization for Continual Learning
• The precise time course of retention.
• Continual Learning Beyond Experience Rehearsal and Full Model Surrogates
• Exemplar-Free Continual Learning for State Space Models
• Sentence Embedding Alignment for Lifelong Relation Extraction
• Human-like Forgetting Curves in Deep Neural Networks
• T2I-ConBench: Text-to-Image Benchmark for Continual Post-training
• Graph Neural Networks with Continual Learning for Fake News Detection from Social Media
• AdaHAT: Adaptive Hard Attention to the Task in Task-Incremental Learning
• A Unified Gradient-based Framework for Task-agnostic Continual Learning-Unlearning
• StPR: Spatiotemporal Preservation and Routing for Exemplar-Free Video Class-Incremental Learning
• Unified Regularity Measures for Sample-wise Learning and Generalization
• Incremental Class Learning using Variational Autoencoders with Similarity Learning
• The cost of learning: Interference effects in memory development.
• Generalization Tower Network: A Novel Deep Neural Network Architecture for Multi-Task Learning
• Adaptive Compression-based Lifelong Learning
• Continual Domain-Tuning for Pretrained Language Models
• Practopoiesis: Or how life fosters a mind
• Transferring Autonomous Driving Knowledge on Simulated and Real\n Intersections
• Overcoming catastrophic forgetting in neural networks
• Contributions by metaplasticity to solving the Catastrophic Forgetting Problem
• Unifying Regularisation Methods for Continual Learning
• Contrastive Supervised Distillation for Continual Representation Learning
• FISH-Tuning: Enhancing PEFT Methods with Fisher Information
• Dynamic Time-aware Continual User Representation Learning
• Design and evolution of modular neural network architectures
• Memory-Modular Classification: Learning to Generalize with Memory Replacement
• Synaptic information transfer in computer models of neocortical columns. [europepmc]
• Letting structure emerge: connectionist and dynamical systems approaches to cognition. [europepmc]
• Effects of aging and IQ on item and associative memory. [europepmc]
• The stability-plasticity dilemma: investigating the continuum from catastrophic forgetting to age-limited learning effects. [europepmc]
• Why won't you do what I want? The informative failures of children and models. [europepmc]
• The cost of learning: interference effects in memory development. [europepmc]
• Neural modularity helps organisms evolve to learn new skills without forgetting old skills. [europepmc]
• Learning structure of sensory inputs with synaptic plasticity leads to interference. [europepmc]
• When Delays Improve Memory: Stabilizing Memory in Children May Require Time. [europepmc]
• Habituation based synaptic plasticity and organismic learning in a quantum perovskite. [europepmc]
• Diffusion-based neuromodulation can eliminate catastrophic forgetting in simple neural networks. [europepmc]
• Comparing continual task learning in minds and machines. [europepmc]
• Computational modeling of interventions for developmental disorders. [europepmc]
• Stability or Plasticity? - A Hierarchical Allostatic Regulation Model of Medial Prefrontal Cortex Function for Social Valuation. [europepmc]
• A Path Toward Explainable AI and Autonomous Adaptive Intelligence: Deep Learning, Adaptive Resonance, and Models of Perception, Emotion, and Action. [europepmc]
• Can sleep protect memories from catastrophic forgetting? [europepmc]
• Brain-inspired replay for continual learning with artificial neural networks. [europepmc]
• Engrams of Fast Learning. [europepmc]
• Replay in Deep Learning: Current Approaches and Missing Biological Elements. [europepmc]
• An exploration of error-driven learning in simple two-layer networks from a discriminative learning perspective. [europepmc]
• Learning in deep neural networks and brains with similarity-weighted interleaved learning. [europepmc]
• Sleep prevents catastrophic forgetting in spiking neural networks by forming a joint synaptic weight representation. [europepmc]
• Dopamine promotes head direction plasticity during orienting movements. [europepmc]
• Neural population dynamics of computing with synaptic modulations. [europepmc]
• Photonic neuromorphic architecture for tens-of-task lifelong learning. [europepmc]

Related

• A model of memory, learning and recognition
• A Neural Network Model of Complementary Learning Systems: Pattern Separation and Completion for Continual Learning
• Overcoming catastrophic forgetting in neural networks
• Memory Networks: Towards Fully Biologically Plausible Learning
• Supervised Generative Reconstruction: An Efficient Way To Flexibly Store and Recognize Patterns