2022/02/09 by Md Kamran Chowdhury Shisher, Shisher, Md Kamran Chowdhury, Tasmeen Zaman Ornee +2
Computer Science · #FOS: Computer and information sciences #Face and Expression Recognition #Information Theory (cs.IT) #Machine Learning (cs.LG) #Neural Networks and Applications #Optical measurement and interference techniques
paper · pdf · doi:10.48550/arxiv.2202.04632
openalex publication_date 2022/02/09 · openalex created_date 2022/04/03 · openalex updated_date 2026/07/28
In this paper, we present a local geometric analysis to interpret how deep feedforward neural networks extract low-dimensional features from high-dimensional data. Our study shows that, in a local geometric region, the optimal weight in one layer of the neural network and the optimal feature generated by the previous layer comprise a low-rank approximation of a matrix that is determined by the Bayes action of this layer. This result holds (i) for analyzing both the output layer and the hidden layers of the neural network, and (ii) for neuron activation functions with non-vanishing gradients. We use two supervised learning problems to illustrate our results: neural network based maximum likelihood classification (i.e., softmax regression) and neural network based minimum mean square estimation. Experimental validation of these theoretical results will be conducted in our future work.