Backpropagation through time and the brain

2019/03/06 by Timothy Lillicrap, Timothy P Lillicrap, Adam Santoro · 3 citations
Computer Science · Engineering · Neuroscience · #Advanced Memory and Neural Computing #Neural Networks and Applications #Neural dynamics and brain function

paper · pdf · doi:10.1016/j.conb.2019.01.011

openalex publication_date 2019/03/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/02

Abstract

It has long been speculated that the backpropagation-of-error algorithm (backprop) may be a model of how the brain learns. Backpropagation-through-time (BPTT) is the canonical temporal-analogue to backprop used to assign credit in recurrent neural networks in machine learning, but there's even less conviction about whether BPTT has anything to do with the brain. Even in machine learning the use of BPTT in classic neural network architectures has proven insufficient for some challenging temporal credit assignment (TCA) problems that we know the brain is capable of solving. Nonetheless, recent work in machine learning has made progress in solving difficult TCA problems by employing novel memory-based and attention-based architectures and algorithms, some of which are brain inspired. Importantly, these recent machine learning methods have been developed in the context of, and with reference to BPTT, and thus serve to strengthen BPTT's position as a useful normative guide for thinking about temporal credit assignment in artificial and biological systems alike.

Citations

• On the difficulty of training Recurrent Neural Networks
• Random feedback weights support learning in deep neural networks
• A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
• Long Short-Term Memory
• Finding Structure in Time
• Exact solutions to the nonlinear dynamics of learning in deep linear neural networks
• Neural Machine Translation by Jointly Learning to Align and Translate
• Unitary Evolution Recurrent Neural Networks
• Associative Long Short-Term Memory
• WaveNet: A Generative Model for Raw Audio
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes
• Categorical Reparameterization with Gumbel-Softmax
• The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
• Unbiased Online Recurrent Optimization
• Cortical microcircuits as gated-recurrent neural networks
• The Kanerva Machine: A Generative Distributed Memory
• Unsupervised Predictive Memory in a Goal-Directed Agent
• Approximating Real-Time Recurrent Learning with Random Kronecker Factors
• RUDDER: Return Decomposition for Delayed Rewards
• Sparse Attentive Backtracking: Temporal CreditAssignment Through Reminding
• Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
• Exploiting Generative AI to Scale up Intelligent Tutoring Systems

Cited by

• Learning Numeracy: Binary Arithmetic with Neural Turing Machines
• The Why, What, and How of Artificial General Intelligence Chip Development
• Enabling Spike-Based Backpropagation for Training Deep Neural Network Architectures

Related

• Can Biologically Plausible Temporal Credit Assignment Rules Match BPTT for Neural Similarity? E-prop as an Example
• Sparse Attentive Backtracking: Temporal CreditAssignment Through Reminding
• Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment
• Sparse Attentive Backtracking: Long-Range Credit Assignment in Recurrent Networks
• Kickback cuts Backprop's red-tape: Biologically plausible credit assignment in neural networks