Deep neural networks from the perspective of ergodic theory

2023/08/04 by Zhang, Fan · 1 citation
#Disordered Systems and Neural Networks (cond-mat.dis-nn) #FOS: Computer and information sciences #FOS: Physical sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2308.03888

Abstract

The design of deep neural networks remains somewhat of an art rather than precise science. By tentatively adopting ergodic theory considerations on top of viewing the network as the time evolution of a dynamical system, with each layer corresponding to a temporal instance, we show that some rules of thumb, which might otherwise appear mysterious, can be attributed heuristics.

Cited by

• Description of the Training Process of Neural Networks via Ergodic Theorem : Ghost nodes

Related

• Deep limits and cut-off phenomena for neural networks
• Description of the Training Process of Neural Networks via Ergodic Theorem : Ghost nodes
• Neural networks: from the perceptron to deep nets
• Periodic Spectral Ergodicity: A Complexity Measure for Deep Neural\n Networks and Neural Architecture Search
• A Survey: Time Travel in Deep Learning Space: An Introduction to Deep Learning Models and How Deep Learning Models Evolved from the Initial Ideas