The energy landscape of a simple neural network

2017/06/21 by Anthony Collins Gamst, Gamst, Anthony Collins, Alden Walker +1
Computer Science · Mathematics · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1706.07101

17 pages, 15 figures

arxiv created 2017/06/21 · arxiv updated 2017/06/23

Abstract

We explore the energy landscape of a simple neural network. In particular, we expand upon previous work demonstrating that the empirical complexity of fitted neural networks is vastly less than a naive parameter count would suggest and that this implicit regularization is actually beneficial for generalization from fitted models.

Related

• On Energy-Based Models with Overparametrized Shallow Neural Networks
• Thermodynamic bounds on energy use in quasi-static Deep Neural Networks
• A learning rule balancing energy consumption and information maximization in a feed-forward neuronal network
• Sampling the space of solutions of an artificial neural network
• Neural networks: from the perceptron to deep nets