2009/07/22 by Wei Lin, Guanrong Chen
Computer Science · Physics and Astronomy · #Model Reduction and Neural Networks #Neural Networks and Applications #Neural Networks and Reservoir Computing
paper · doi:10.1109/tnn.2009.2024148
openalex publication_date 2009/07/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29
In the literature, it was reported that the chaotic artificial neural network model with sinusoidal activation functions possesses a large memory capacity as well as a remarkable ability of retrieving the stored patterns, better than the conventional chaotic model with only monotonic activation functions such as sigmoidal functions. This paper, from the viewpoint of the anti-integrable limit, elucidates the mechanism inducing the superiority of the model with periodic activation functions that includes sinusoidal functions. Particularly, by virtue of the anti-integrable limit technique, this paper shows that any finite-dimensional neural network model with periodic activation functions and properly selected parameters has much more abundant chaotic dynamics that truly determine the model's memory capacity and pattern-retrieval ability. To some extent, this paper mathematically and numerically demonstrates that an appropriate choice of the activation functions and control scheme can lead to a large memory capacity and better pattern-retrieval ability of the artificial neural network models.