2018/02/02 by Claudio Gallicchio, Gallicchio, Claudio · 8 citations
Computer Science · Engineering · Mathematics · Neuroscience · #Advanced Memory and Neural Computing #Artificial Intelligence (cs.AI) #Artificial intelligence #Artificial neural network #Computer science #Context (archaeology) #Deep learning #Dynamical Systems (math.DS) #FOS: Computer and information sciences #FOS: Mathematics #Long short term memory #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine learning #Neural Networks and Reservoir Computing #Neural dynamics and brain function #Recurrent neural network #Reservoir computing #Term (time) #cs.AI #cs.LG #math.DS #stat.ML
paper · pdf · doi:10.48550/arxiv.1802.00748
published in arXiv (Cornell University) (Cornell University) · This is a pre-print (pre-review) version of the paper accepted for presentation at the 26th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), Bruges (Belgium), 25-27 April 2018
arxiv created 2018/02/02 · openalex publication_date 2018/02/02 · arxiv updated 2018/02/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
The extension of deep learning towards temporal data processing is gaining an increasing research interest. In this paper we investigate the properties of state dynamics developed in successive levels of deep recurrent neural networks (RNNs) in terms of short-term memory abilities. Our results reveal interesting insights that shed light on the nature of layering as a factor of RNN design. Noticeably, higher layers in a hierarchically organized RNN architecture results to be inherently biased towards longer memory spans even prior to training of the recurrent connections. Moreover, in the context of Reservoir Computing framework, our analysis also points out the benefit of a layered recurrent organization as an efficient approach to improve the memory skills of reservoir models.