How deep learning works --The geometry of deep learning

2017/10/30 by Xiao Dong, Dong, Xiao, Jiasong Wu +3 · 1 citation
Computer Science · #Advanced Graph Neural Networks #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Stochastic Gradient Optimization Techniques #Topological and Geometric Data Analysis

paper · pdf · doi:10.48550/arxiv.1710.10784

openalex publication_date 2017/10/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Why and how that deep learning works well on different tasks remains a mystery from a theoretical perspective. In this paper we draw a geometric picture of the deep learning system by finding its analogies with two existing geometric structures, the geometry of quantum computations and the geometry of the diffeomorphic template matching. In this framework, we give the geometric structures of different deep learning systems including convolutional neural networks, residual networks, recursive neural networks, recurrent neural networks and the equilibrium prapagation framework. We can also analysis the relationship between the geometrical structures and their performance of different networks in an algorithmic level so that the geometric framework may guide the design of the structures and algorithms of deep learning systems.

Citations

• Reinforcement Learning Neural Turing Machines - Revised
• Exponential expressivity in deep neural networks through transient chaos

Cited by

• Spacetime as the optimal generative network of quantum states: a roadmap to QM=GR?

Related

• Geometrization of deep networks for the interpretability of deep learning systems
• Mathematical Foundations of Geometric Deep Learning
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• Geometric Deep Learning: Going beyond Euclidean data
• Geometric Understanding of Deep Learning