Expanding memory in recurrent spiking networks

2023/10/29 by Ismael Balafrej, Fabien Alibart, Balafrej, Ismael +3
Computer Science · Engineering · Neuroscience · #Advanced Memory and Neural Computing #FOS: Computer and information sciences #Neural Networks and Reservoir Computing #Neural and Evolutionary Computing (cs.NE) #Neural dynamics and brain function

paper · pdf · doi:10.48550/arxiv.2310.19067

openalex publication_date 2023/10/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recurrent spiking neural networks (RSNNs) are notoriously difficult to train because of the vanishing gradient problem that is enhanced by the binary nature of the spikes. In this paper, we review the ability of the current state-of-the-art RSNNs to solve long-term memory tasks, and show that they have strong constraints both in performance, and for their implementation on hardware analog neuromorphic processors. We present a novel spiking neural network that circumvents these limitations. Our biologically inspired neural network uses synaptic delays, branching factor regularization and a novel surrogate derivative for the spiking function. The proposed network proves to be more successful in using the recurrent connections on memory tasks.

Related

• Effective and Efficient Computation with Multiple-timescale Spiking Recurrent Neural Networks
• Long Short-Term Memory Spiking Networks and Their Applications
• Conversion of Artificial Recurrent Neural Networks to Spiking Neural Networks for Low-power Neuromorphic Hardware
• Mapping high-performance RNNs to in-memory neuromorphic chips
• A Scalable Hybrid Training Approach for Recurrent Spiking Neural Networks