Analysis on the Nonlinear Dynamics of Deep Neural Networks: Topological Entropy and Chaos

2018/04/03 by Husheng Li, Li, Husheng
Computer Science · Neuroscience · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications #Neural Networks and Reservoir Computing #Neural dynamics and brain function

paper · pdf · doi:10.48550/arxiv.1804.03987

openalex publication_date 2018/04/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The theoretical explanation for deep neural network (DNN) is still an open problem. In this paper DNN is considered as a discrete-time dynamical system due to its layered structure. The complexity provided by the nonlinearity in the dynamics is analyzed in terms of topological entropy and chaos characterized by Lyapunov exponents. The properties revealed for the dynamics of DNN are applied to analyze the corresponding capabilities of classification and generalization.

Citations

• Random dynamical systems
• Fractal Geometry: Mathematical Foundations and Applications.
• Understanding deep convolutional networks

Related

• Depth-Width Trade-offs for Neural Networks via Topological Entropy
• Finite-time Lyapunov exponents of deep neural networks
• Lyapunov spectra of chaotic recurrent neural networks
• Deep Learning of Chaos Classification
• The Expressivity and Training of Deep Neural Networks: toward the Edge of Chaos?