Toward Practical Equilibrium Propagation: Brain-inspired Recurrent Neural Network with Feedback Regulation and Residual Connections

2025/08/05 by Zhuo Liu, Tao Chen, Liu, Zhuo +1 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Biological sciences #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE) #Neurons and Cognition (q-bio.NC)

paper · pdf · doi:10.48550/arxiv.2508.11659

openalex publication_date 2025/08/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Brain-like intelligent systems need brain-like learning methods. Equilibrium Propagation (EP) is a biologically plausible learning framework with strong potential for brain-inspired computing hardware. However, existing im-plementations of EP suffer from instability and prohibi-tively high computational costs. Inspired by the structure and dynamics of the brain, we propose a biologically plau-sible Feedback-regulated REsidual recurrent neural network (FRE-RNN) and study its learning performance in EP framework. Feedback regulation enables rapid convergence by reducing the spectral radius. The improvement in con-vergence property reduces the computational cost and train-ing time of EP by orders of magnitude, delivering perfor-mance on par with backpropagation (BP) in benchmark tasks. Meanwhile, residual connections with brain-inspired topologies help alleviate the vanishing gradient problem that arises when feedback pathways are weak in deep RNNs. Our approach substantially enhances the applicabil-ity and practicality of EP in large-scale networks that un-derpin artificial intelligence. The techniques developed here also offer guidance to implementing in-situ learning in physical neural networks.

Citations

• Biologically plausible models of cognitive flexibility: merging recurrent neural networks with full-brain dynamics
• Energy-based learning algorithms for analog computing: a comparative study
• Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment
• Improving equilibrium propagation without weight symmetry through Jacobian homeostasis
• Backpropagation at the Infinitesimal Inference Limit of Energy-Based Models: Unifying Predictive Coding, Equilibrium Propagation, and Contrastive Hebbian Learning
• Learning on Arbitrary Graph Topologies via Predictive Coding
• Latent Equilibrium: A unified learning theory for arbitrarily fast computation with arbitrarily slow neurons
• Credit Assignment in Neural Networks through Deep Feedback Control
• Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing\n its Gradient Estimator Bias
• Equilibrium Propagation with Continual Weight Updates
• Updates of Equilibrium Prop Match Gradients of Backprop Through Time in an RNN with Static Input
• Hebbian-Descent
• Dendritic cortical microcircuits approximate the backpropagation\n algorithm
• Generalization of Equilibrium Propagation to Vector Field Dynamics
• Biologically Motivated Algorithms for Propagating Local Target Representations
• Stable Recurrent Models
• Learning to Adapt by Minimizing Discrepancy
• Attention Is All You Need
• Direct Feedback Alignment Provides Learning in Deep Neural Networks
• Equilibrium Propagation: Bridging the Gap Between Energy-Based Models\n and Backpropagation
• Deep Residual Learning for Image Recognition
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• ImageNet classification with deep convolutional neural networks
• Definition and properties of Lagrangian coherent structures from finite-time Lyapunov exponents in two-dimensional aperiodic flows
• Determining Lyapunov exponents from a time series

Cited by

• Scalable Equilibrium Propagation via Intermediate Error Signals for Deep Convolutional CRNNs

Related

• Scaling Equilibrium Propagation to Deeper Neural Network Architectures
• Equilibrium Propagation for Complete Directed Neural Networks
• Continual Weight Updates and Convolutional Architectures for Equilibrium Propagation
• Equilibrium Propagation with Continual Weight Updates
• Updates of Equilibrium Prop Match Gradients of Backprop Through Time in an RNN with Static Input