vix.ing · top · new · best · stats

Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges

2021/04/27 by Michael M. Bronstein, Bronstein, Michael M., Joan Bruna +5 · 7 voices · 561 citations
Computer Science · Mathematics · #Advanced Graph Neural Networks #Artificial Intelligence (cs.AI) #Artificial intelligence #Artificial neural network #Computational Geometry (cs.CG) #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Curse of dimensionality #Deep learning #FOS: Computer and information sciences #Feature learning #Graph Theory and Algorithms #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine learning #Simple (philosophy) #Theoretical computer science #Topological and Geometric Data Analysis #Unification #cs.AI #cs.CG #cs.CV #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2104.13478

published in arXiv (Cornell University) (Cornell University) · 156 pages. Work in progress -- comments welcome!

openalex publication_date 2021/04/27 · arxiv published 2021/04/27 · arxiv created 2021/05/02 · arxiv updated 2021/05/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/02

Abstract

The last decade has witnessed an experimental revolution in data science and machine learning, epitomised by deep learning methods. Indeed, many high-dimensional learning tasks previously thought to be beyond reach -- such as computer vision, playing Go, or protein folding -- are in fact feasible with appropriate computational scale. Remarkably, the essence of deep learning is built from two simple algorithmic principles: first, the notion of representation or feature learning, whereby adapted, often hierarchical, features capture the appropriate notion of regularity for each task, and second, learning by local gradient-descent type methods, typically implemented as backpropagation. While learning generic functions in high dimensions is a cursed estimation problem, most tasks of interest are not generic, and come with essential pre-defined regularities arising from the underlying low-dimensionality and structure of the physical world. This text is concerned with exposing these regularities through unified geometric principles that can be applied throughout a wide spectrum of applications. Such a 'geometric unification' endeavour, in the spirit of Felix Klein's Erlangen Program, serves a dual purpose: on one hand, it provides a common mathematical framework to study the most successful neural network architectures, such as CNNs, RNNs, GNNs, and Transformers. On the other hand, it gives a constructive procedure to incorporate prior physical knowledge into neural architectures and provide principled way to build future architectures yet to be invented.

Citations

Cited by

Discussions

Related