Linear Algebra and Duality of Neural Networks

2018/09/12 by Georgiev, Galin
#Computer Vision and Pattern Recognition (cs.CV) #E.0 #FOS: Computer and information sciences #G.1.3 #G.1.6 #I.2.0

paper · doi:10.48550/arxiv.1809.04711

Abstract

Bases, mappings, projections and metrics, natural for Neural network training, are introduced. Graph-theoretical interpretation is offered. Non-Gaussianity naturally emerges, even in relatively simple datasets. Training statistics, hierarchies and energies are analyzed, from physics point of view. Duality between observables (for example, pixels) and observations is established. Relationship between exact and numerical solutions is studied. Physics and financial mathematics interpretations of a key problem are offered. Examples support all new concepts.

Related

• Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't
• TASI Lectures on Physics for Machine Learning
• Geometry Perspective Of Estimating Learning Capability Of Neural Networks
• Lecture Notes on Linear Neural Networks: A Tale of Optimization and Generalization in Deep Learning
• Neural Networks