Analysis on the Nonlinear Dynamics of Deep Neural Networks: Topological Entropy and Chaos

2018/04/03 by Husheng Li, Li, Husheng · 8 citations
Computer Science · Mathematics · Neuroscience · #Artificial intelligence #Artificial neural network #CHAOS (operating system) #Chaotic #Computer science #Entropy (arrow of time) #FOS: Computer and information sciences #Generalization #Lyapunov exponent #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Mathematical analysis #Mathematics #Neural Networks and Applications #Neural Networks and Reservoir Computing #Neural dynamics and brain function #Nonlinear system #Physics #Pure mathematics #Statistical physics #Topological dynamics #Topological entropy #Topology (electrical circuits) #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1804.03987

published in arXiv (Cornell University) (Cornell University) · 17 pages, 24 figures

openalex publication_date 2018/04/03 · arxiv created 2019/01/08 · arxiv updated 2019/01/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The theoretical explanation for deep neural network (DNN) is still an open problem. In this paper DNN is considered as a discrete-time dynamical system due to its layered structure. The complexity provided by the nonlinearity in the dynamics is analyzed in terms of topological entropy and chaos characterized by Lyapunov exponents. The properties revealed for the dynamics of DNN are applied to analyze the corresponding capabilities of classification and generalization.

Citations

• Random dynamical systems
• Fractal Geometry: Mathematical Foundations and Applications.
• Understanding Deep Convolutional Networks

Related

• Topological and Dynamical Complexity of Random Neural Networks
• Depth-Width Trade-offs for Neural Networks via Topological Entropy
• Finite-time Lyapunov exponents of deep neural networks
• Lyapunov spectra of chaotic recurrent neural networks
• Deep Learning of Chaos Classification