Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
2021/04/27 by Michael M. Bronstein, Bronstein, Michael M., Joan Bruna +5 · 7 voices · 561 citations
Computer Science · Mathematics · #Advanced Graph Neural Networks #Artificial Intelligence (cs.AI) #Artificial intelligence #Artificial neural network #Computational Geometry (cs.CG) #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Curse of dimensionality #Deep learning #FOS: Computer and information sciences #Feature learning #Graph Theory and Algorithms #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine learning #Simple (philosophy) #Theoretical computer science #Topological and Geometric Data Analysis #Unification #cs.AI #cs.CG #cs.CV #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.2104.13478
published in arXiv (Cornell University) (Cornell University) · 156 pages. Work in progress -- comments welcome!
openalex publication_date 2021/04/27 · arxiv published 2021/04/27 · arxiv created 2021/05/02 · arxiv updated 2021/05/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/02
Abstract
The last decade has witnessed an experimental revolution in data science and machine learning, epitomised by deep learning methods. Indeed, many high-dimensional learning tasks previously thought to be beyond reach -- such as computer vision, playing Go, or protein folding -- are in fact feasible with appropriate computational scale. Remarkably, the essence of deep learning is built from two simple algorithmic principles: first, the notion of representation or feature learning, whereby adapted, often hierarchical, features capture the appropriate notion of regularity for each task, and second, learning by local gradient-descent type methods, typically implemented as backpropagation. While learning generic functions in high dimensions is a cursed estimation problem, most tasks of interest are not generic, and come with essential pre-defined regularities arising from the underlying low-dimensionality and structure of the physical world. This text is concerned with exposing these regularities through unified geometric principles that can be applied throughout a wide spectrum of applications. Such a 'geometric unification' endeavour, in the spirit of Felix Klein's Erlangen Program, serves a dual purpose: on one hand, it provides a common mathematical framework to study the most successful neural network architectures, such as CNNs, RNNs, GNNs, and Transformers. On the other hand, it gives a constructive procedure to incorporate prior physical knowledge into neural architectures and provide principled way to build future architectures yet to be invented.
Citations
Cited by
- Implementations of Quantum and Classical Topology-Aligned Architectures for Molecular Property Prediction
- Parallelisation of Discrete Exterior Calculus via Representation Theory on Curved and Three-Dimensional Meshes
- Graph Neural Networks for Predicting Solvability of Finite Groups
- Discovering Symmetry Groups with Flow Matching
- Photonics-Enhanced Graph Convolutional Networks
- Beyond Accuracy: A Geometric Stability Analysis of Large Language Models in Chess Evaluation
- Machine learning for RNA-targeting drug design
- Beyond MMD: Evaluating Graph Generative Models with Geometric Deep Learning
- Neuromodulation-inspired gated associative memory networks:extended memory retrieval and emergent multistability
- Neural quantum states for entanglement depth certification from randomized Pauli measurements
- Torch Geometric Pool: the PyTorch library for pooling in Graph Neural Networks
- GoMS: Graph of Molecule Substructure Network for Molecule Property Prediction
- Bridging Data and Physics: A Graph Neural Network-Based Hybrid Twin Framework
- Generalized Spherical Neural Operators: Green's Function Formulation
- Supervised Learning of Random Neural Architectures Structured by Latent Random Fields on Compact Boundaryless Multiply-Connected Manifolds
- Generative Modeling of Entangled Polymers with a Distance-Based Variational Autoencoder
- Drawback of Enforcing Equivariance and its Compensation via the Lens of Expressive Power
- RF sensing with dense IoT network graphs: An EM-informed analysis
- Learning the Cosmic Web: Graph-based Classification of Simulated Galaxies by their Dark Matter Environments
- PERM EQ x GRAPH EQ: Equivariant Neural Networks for Quantum Molecular Learning
- Achieving Approximate Symmetry Is Exponentially Easier than Exact Symmetry
- Generalization Beyond Benchmarks: Evaluating Learnable Protein-Ligand Scoring Functions on Unseen Targets
- On the Temporality for Sketch Representation Learning
- Partially Equivariant Reinforcement Learning in Symmetry-Breaking Environments
- Symmetries at the origin of hierarchical emergence
- Beyond Topology: A Morphological Symmetry Graph Representation for Locomotion Policy Learning
- A Differentiable Measure of Algebraic Complexity: Provably Exact Discovery of Group Structures
- Accumulated Local Effects and Graph Neural Networks for link prediction
- The Intrinsic Dimension of Collider Events and Model-Independent Searches in 100 Dimensions
- The Alexander-Hirschowitz theorem for neurovarieties
- Categorical Equivariant Deep Learning: Category-Equivariant Neural Networks and Universal Approximation Theorems
- Bridging Philosophy and Machine Learning: A Structuralist Framework for Classifying Neural Network Representations
- Gauge-Equivariant Graph Networks via Self-Interference Cancellation
- Tensor Gauge Flow Models
- Warm-starting active-set solvers using graph neural networks
- Lost in Serialization: Invariance and Generalization of LLM Graph Reasoners
- TomoGraphView: 3D Medical Image Classification with Omnidirectional Slice Representations and Graph Neural Networks
- Generalizable Insights for Graph Transformers in Theory and Practice
- Recurrent Equivariant Constraint Modulation: Learning Per-Layer Symmetry Relaxation from Data
- Context-aware Learned Mesh-based Simulation via Trajectory-Level Meta-Learning
- Twirlator: A Pipeline for Analyzing Subgroup Symmetry Effects in Quantum Machine Learning Ansatzes
- Shaping manifolds in equivariant recurrent neural networks
- Sparse, self-organizing ensembles of local kernels detect rare statistical anomalies
- Scalable Single-Cell Gene Expression Generation with Latent Diffusion Models
- Learning with Category-Equivariant Architectures for Human Activity Recognition
- Geometry as a Missing Axis of Representation Quality: The Variational Geometric Information Bottleneck under Data Scarcity
- Learning Geometry: A Framework for Building Adaptive Manifold Models through Metric Optimization
- Power spectrum signatures of graphs
- Shape-Based Inductive Bias for Glioma Grading from Tumor Contours
- Rotational equivariance and locality in data-driven subgrid-scale closures
- Coalgebras for categorical deep learning: Representability and universal approximation
- The Neural Differential Manifold: An Architecture with Explicit Geometric Structure
- Equivariant Neural Networks for General Linear Symmetries on Lie Algebras
- A roadmap for curvature-based geometric data analysis and learning
- Hierarchical Graph Networks for Accurate Weather Forecasting via Lightweight Training
- HollowFlow: Efficient Sample Likelihood Evaluation using Hollow Message Passing
- On Uncertainty Calibration for Equivariant Functions
- Bayesian Inference of Primordial Magnetic Field Parameters from CMB with Spherical Graph Neural Networks
- There is No "apple" in Timeseries: Rethinking TSFM through the Lens of Invariance
- On Universality of Deep Equivariant Networks
- Conditional Clifford-Steerable CNNs with Complete Kernel Basis for PDE Modeling
- BACE: Behavior-Adaptive Connectivity Estimation for Interpretable Graphs of Neural Dynamics
- Probabilistic Hyper-Graphs using Multiple Randomly Masked Autoencoders for Semi-supervised Multi-modal Multi-task Learning
- Inverse-Free Wilson Loops for Transformers: A Practical Diagnostic for Invariance and Order Sensitivity
- MaNGO - Adaptable Graph Network Simulators via Meta-Learning
- When Does Global Attention Help? A Unified Empirical Study on Atomistic Graph Learning
- Fundamental Limits of Crystalline Equivariant Graph Neural Networks: A Circuit Complexity Perspective
- ATOM: A Pretrained Neural Operator for Multitask Molecular Dynamics
- ResCP: Reservoir Conformal Prediction for Time Series Forecasting
- The Role of Feature Interactions in Graph-based Tabular Deep Learning
- Toward a Unified Geometry Understanding: Riemannian Diffusion Framework for Graph Generation and Prediction
- Curvature-Aware Deep Learning for Vector Boson Fusion: Differential Geometry, Physics-Inspired Features, and Quantum Method Limitations
- Flock: A Knowledge Graph Foundation Model via Learning on Random Walks
- The Principle of Isomorphism: A Theory of Population Activity in Grid Cells and Beyond
- VDW-GNNs: Vector diffusion wavelets for geometric graph neural networks
- AlphaSAGE: Structure-Aware Alpha Mining via GFlowNets for Robust Exploration
- Single-Core Superscalar Optimization of Clifford Neural Layers
- FS-KAN: Permutation Equivariant Kolmogorov-Arnold Networks via Function Sharing
- Adaptive Canonicalization with Application to Invariant Anisotropic Geometric Networks
- ADAPT: Lightweight, Long-Range Machine Learning Force Fields Without Graphs
- High Luminosity LHC data collected by CMS experiment -- an excellent ground for the search of Rare Radiative Bs0 meson decays: A Review
- From Neural Networks to Logical Theories: The Correspondence between Fibring Modal Logics and Fibring Neural Networks
- Identifiable Equivariant Networks are Layerwise Equivariant
- Interpretable Discovery of One-parameter Subgroups: A Modular Framework for Elliptical, Hyperbolic, and Parabolic Symmetries
- Learning Equivariant Functions via Quadratic Forms
- The role of fibration symmetries in geometric deep learning
- Geometric Autoencoder Priors for Bayesian Inversion: Learn First Observe Later
- Learning Coherent Representations: A Topological Approach to Interpretability
- Spontaneous symmetry breaking and Goldstone modes for deep information propagation
- Descriptive power and predictive limits of a discrete Hasimoto--DNLS model of protein backbone structure
- Hybrid Lie semi-group and cascade structures for the generalized Gaussian derivative model for visual receptive fields
- Hierarchical Self-Attention: Generalizing Neural Attention Mechanics to Multi-Scale Problems
- Floating-Body Hydrodynamic Neural Networks
- Local-Canonicalization Equivariant Graph Neural Networks for Sample-Efficient and Generalizable Swarm Robot Control
- SPARK: Adaptive Low-Rank Knowledge Graph Modeling in Hybrid Geometric Spaces for Recommendation
- Geometrically Constrained and Token-Based Probabilistic Spatial Transformers
- Optimal message passing for molecular prediction is simple, attentive and spatial
- Graph-based Integrated Gradients for Explaining Graph Neural Networks
- Data-driven discovery of dynamical models in biology
- Learning from one graph: transductive learning guarantees via the geometry of small random worlds
- Learning words in groups: fusion algebras, tensor ranks and grokking
- Data-Efficient Time-Dependent PDE Surrogates: Graph Neural Simulators vs. Neural Operators
- Contextuality, Holonomy and Discrete Fiber Bundles in Group-Valued Boltzmann Machines
- Topotein: Topological Deep Learning for Protein Representation Learning
- Learning functions through Diffusion Maps
- RotaTouille: Rotation Equivariant Deep Learning for Contours
- Quantum-Enhanced Optimization by Warm Starts
- On sources to variabilities of simple cells in the primary visual cortex: A principled theory for the interaction between geometric image transformations and receptive field responses
- A gauge theory of complex adaptive systems
- Universal Representation of Generalized Convex Functions and their Gradients
- Neural Field Turing Machine: A Differentiable Spatial Computer
- Memorization in Graph Neural Networks
- DegDiT: Controllable Audio Generation with Dynamic Event Graph Guided Diffusion Transformer
- Graph Concept Bottleneck Models
- Symmetry-Constrained Multi-Scale Physics-Informed Neural Networks for Graphene Electronic Band Structure Prediction
- Exact Verification of Graph Neural Networks with Incremental Constraint Solving
- TANGO: Graph Neural Dynamics via Learned Energy and Tangential Flows
- Quantum Spectral Reasoning: A Non-Neural Architecture for Interpretable Machine Learning
- The Geometry of Cortical Computation: Manifold Disentanglement and Predictive Dynamics in VCNet
- Adaptive Riemannian Graph Neural Networks
- Categorical Construction of Logically Verifiable Neural Architectures
- PyG 2.0: Scalable Learning on Real World Graphs
- Foundations of Interpretable Models
- Light-Weight Diffusion Multiplier and Uncertainty Quantification for Fourier Neural Operators
- Hierarchical Message-Passing Policies for Multi-Agent Reinforcement Learning
- Towards a Robust Transport Network With Self-adaptive Network Digital Twin
- Geometry of Neural Reinforcement Learning in Continuous State and Action Spaces
- NAICS-Aware Graph Neural Networks for Large-Scale POI Co-visitation Prediction: A Multi-Modal Dataset and Methodology
- Flow Equivariant Recurrent Neural Networks
- Partial Symmetry Enforced Attention Decomposition (PSEAD): A Group-Theoretic Framework for Equivariant Transformers in Biological Systems
- GraphVSSM: Graph Variational State-Space Model for Probabilistic Spatiotemporal Inference of Dynamic Exposure and Vulnerability for Regional Disaster Resilience Assessment
- Localized FNO for Spatiotemporal Hemodynamic Upsampling in Aneurysm MRI
- Binarizing Physics-Inspired GNNs for Combinatorial Optimization
- Tensor-Tensor Products, Group Representations, and Semidefinite Programming
- Emergence of Quantised Representations Isolated to Anisotropic Functions
- DualEquiNet: A Dual-Space Hierarchical Equivariant Network for Large Biomolecules
- HSG-12M: A Large-Scale Benchmark of Spatial Multigraphs from the Energy Spectra of Non-Hermitian Crystals
- Test-Time Canonicalization by Foundation Models for Robust Perception
- Polaritonic Machine Learning for Graph-based Data Analysis
- Do we need equivariant models for molecule generation?
- The Riemannian Geometry associated to Gradient Flows of Linear Convolutional Networks
- Sinkhorn Normalization of Diffusion Kernels
- Relational inductive biases on attention mechanisms
- CosmoBench: A Multiscale, Multiview, Multitask Cosmology Benchmark for Geometric Deep Learning
- GradMetaNet: An Equivariant Architecture for Learning on Gradients
- Reinforcement Learning for Automated Cybersecurity Penetration Testing
- FoGE: Fock Space inspired encoding for graph prompting
- Fast and Distributed Equivariant Graph Neural Networks by Virtual Node Learning
- Ontology Neural Network and ORTSF: A Framework for Topological Reasoning and Delay-Robust Control
- Physics-Guided Radiotherapy Treatment Planning with Deep Learning
- Algebraic Priors for Approximately Equivariant Networks
- On Equivariant Model Selection through the Lens of Uncertainty
- Robot Tactile Gesture Recognition Based on Full-body Modular E-skin
- The Relationship between Cognition and Computation: "Global-first" Cognition versus Local-first Computation
- Relational Deep Learning: Challenges, Foundations and Next-Generation Architectures
- Geometric deep learning assists protein engineering. Opportunities and Challenges
- CP2: Leveraging Geometry for Conformal Prediction via Canonicalization
- Sampling 3D Molecular Conformers with Diffusion Transformers
- Over-squashing in Spatiotemporal Graph Neural Networks
- Embedding physical symmetries into machine-learned reduced plasma physics models via data augmentation
- Uncertainty-Aware Graph Neural Networks: A Multi-Hop Evidence Fusion Approach
- Symmetry in Neural Network Parameter Spaces
- Learning geometric invariant features for classification of vector polygons with graph message-passing neural network
- Taxonomy of reduction matrices for Graph Coarsening
- Quantum-Inspired Differentiable Integral Neural Networks (QIDINNs): A Feynman-Based Architecture for Continuous Learning Over Streaming Data
- Principled Approaches for Extending Neural Architectures to Function Spaces for Operator Learning
- Shapley Machine: A Game-Theoretic Framework for N-Agent Ad Hoc Teamwork
- Demystifying Topological Message-Passing with Relational Structures: A Case Study on Oversquashing in Simplicial Message-Passing
- Graph Persistence goes Spectral
- Mitigating Degree Bias Adaptively with Hard-to-Learn Nodes in Graph Contrastive Learning
- Morphologically Symmetric Reinforcement Learning for Ambidextrous Bimanual Manipulation
- Hamiltonian Normalizing Flows as kinetic PDE solvers: application to the 1D Vlasov-Poisson Equations
- Clique detection using symmetry-restricted quantum circuits
- ToothForge: Automatic Dental Shape Generation using Synchronized Spectral Embeddings
- Equivariant Subgraph Aggregation Networks
- Unwrapping photonic reservoirs: enhanced expressivity via random Fourier encoding over stretched domains
- On Universality Classes of Equivariant Networks
- Protein engineering in the deep learning era
- Data for Towards Improved Quantum Machine Learning for Molecular Force Fields
- AMBER: Adaptive Mesh Generation by Iterative Mesh Resolution Prediction
- The Generalized Skew Spectrum of Graphs
- EquiReg: Equivariance Regularized Diffusion for Inverse Problems
- Symplectic Generative Networks (SGNs): A Hamiltonian Framework for Invertible Deep Generative Modeling
- Assured Autonomy with Neuro-Symbolic Perception
- Lorentz Local Canonicalization: How to Make Any Network Lorentz-Equivariant
- A Framework for Stochastic Differentiable Programming
- Scalable Generation of Spatial Transcriptomics from Histology Images via Whole-Slide Flow Matching
- Message-Passing State-Space Models: Improving Graph Learning with Modern Sequence Modeling
- CoRe-GNN: Multilevel Message passing on Coarsened graphs
- The Discovery Engine: A Framework for AI-Driven Synthesis and Navigation of Scientific Knowledge Landscapes
- Directed Semi-Simplicial Learning with Applications to Brain Activity Decoding
- Backward Oversmoothing: why is it hard to train deep Graph Neural Networks?
- HOPSE: Scalable Higher-Order Positional and Structural Encoder for Combinatorial Representations
- Set-LLM: A Permutation-Invariant LLM
- RECON: Robust symmetry discovery via Explicit Canonical Orientation Normalization
- Symmetry-Breaking Descent for Invariant Cost Functionals
- Surrogate Modeling of 3D Rayleigh-Benard Convection with Equivariant Autoencoders
- Graph Alignment for Benchmarking Graph Neural Networks and Learning Positional Encodings
- AdS-GNN -- a Conformally Equivariant Graph Neural Network
- Solving graph problems using permutation-invariant quantum machine learning
- Continuous Domain Generalization
- Graph and Simplicial Complex Prediction Gaussian Process via the Hodgelet Representations
- Relational Graph Transformer
- GNN-Suite: a Graph Neural Network Benchmarking Framework for Biomedical Informatics
- Quotient Complex Transformer (QCformer) for Perovskite Data Analysis
- Stability of Entropic Wasserstein Barycenters and application to random geometric graphs
- Polarized Target Nuclear Magnetic Resonance Measurements with Deep Neural Networks
- Statistical CSI-Based Distributed Precoding Design for OFDM-Cooperative Multi-Satellite Systems
- Towards a Taxonomy of Graph Learning Datasets
- RISE: Radius of Influence based Subgraph Extraction for 3D Molecular Graph Explanation
- GeloVec: Higher Dimensional Geometric Smoothing for Coherent Visual Feature Extraction in Image Segmentation
- Geometric Causal Models
- MimicBot: Combining Imitation and Reinforcement Learning to win in Bot Bowl
- Efficient Mean Curvature Computation on High-Dimensional Data Manifolds
- Extending Neural Operators: Robust Handling of Functions Beyond the Training Set
- On the Dynamics of Observation and Semantics
- Geometry-Aware Tabular Diffusion
- Geometric Observables for Financial Regime Detection
- Scalable Message-Passing Quantum Graph Neural Networks in the Weisfeiler-Leman Hierarchy
- SFi-Former: Sparse Flow Induced Attention for Graph Transformer
- Concept Graph Convolutions: Message Passing in the Concept Space
- Flow Sampling: Learning to Sample from Unnormalized Densities via Denoising Conditional Processes
- Transcending Dimensions using Generative AI: Real-Time 3D Model Generation in Augmented Reality
- Self-Supervised Graph Neural Networks for Accurate Prediction of Néel Temperature
- Foundations of Equivariant Deep Learning: Unifying Graph and Sheaf Neural Networks
- Data efficiency in graph networks through equivariance
- Dysarthria Normalization via Local Lie Group Transformations for Robust ASR
- Machine Learning is Good for Physics - and Vice Versa
- Position: It's Time to Optimize LLMs for Self-Consistency
- Leveraging Point Transformers for Detecting Anatomical Landmarks in Digital Dentistry
- AtlasD: Automatic Local Symmetry Discovery
- InfoGain Wavelets: Furthering the Design of Graph Diffusion Wavelets
Discussions
- Geometric Deep Learning Grids, Groups, Graphs, Geodesics, and Gauges [pdf] [hn, 3 points, 0 comments]
- Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges [hn, 3 points, 1 comments]
- Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges [hn, 2 points, 0 comments]
- Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges [hn, 2 points, 0 comments]
- Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges (2021) [lobsters, 1 points, 0 comments]
- Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges (2021) https://lobste.rs/s/7npfkd #math #ai [bsky, 0 points, 0 comments]
- arxiv.org/abs/2104.13478 [bsky, 0 points, 0 comments]
Related