2022/06/07 by Jonathan Dong, Erik Börve, Dong, Jonathan +5 · 2 citations
Computer Science · Engineering · #Advanced Memory and Neural Computing #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications #Neural Networks and Reservoir Computing #Neural and Evolutionary Computing (cs.NE)
paper · pdf · doi:10.48550/arxiv.2206.03854
openalex publication_date 2022/06/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Reservoir Computing is a class of Recurrent Neural Networks with internal weights fixed at random. Stability relates to the sensitivity of the network state to perturbations. It is an important property in Reservoir Computing as it directly impacts performance. In practice, it is desirable to stay in a stable regime, where the effect of perturbations does not explode exponentially, but also close to the chaotic frontier where reservoir dynamics are rich. Open questions remain today regarding input regularization and discontinuous activation functions. In this work, we use the recurrent kernel limit to draw new insights on stability in reservoir computing. This limit corresponds to large reservoir sizes, and it already becomes relevant for reservoirs with a few hundred neurons. We obtain a quantitative characterization of the frontier between stability and chaos, which can greatly benefit hyperparameter tuning. In a broader sense, our results contribute to understanding the complex dynamics of Recurrent Neural Networks.