A Practical Method for Constructing Equivariant Multilayer Perceptrons\n for Arbitrary Matrix Groups

2021/04/19 by Marc Finzi, Max Welling, Finzi, Marc +3 · 25 citations
Computer Science · Mathematics · #Advanced Graph Neural Networks #Algebra over a field #Algorithm #Artificial intelligence #Artificial neural network #Combinatorics #Computer science #Construct (python library) #Dynamical Systems (math.DS) #Equivariant map #FOS: Computer and information sciences #FOS: Mathematics #Generalization #Geometry #Group (periodic table) #Homogeneous space #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Materials science #Mathematical analysis #Mathematics #Matrix (chemical analysis) #Matrix Theory and Algorithms #Neural Networks and Applications #Perceptron #Physics #Pure mathematics #Rotation matrix #Topic Modeling #Translation (biology) #cs.LG #math.DS #stat.ML

paper · pdf · doi:10.48550/arxiv.2104.09459

published in arXiv (Cornell University) (Cornell University) · Library: https://github.com/mfinzi/equivariant-MLP, Documentation: https://emlp.readthedocs.io/en/latest/, Examples: https://colab.research.google.com/github/mfinzi/equivariant-MLP/blob/master/docs/notebooks/colabs/all.ipynb

arxiv created 2021/04/19 · openalex publication_date 2021/04/19 · arxiv updated 2021/04/20 · openalex created_date 2021/06/22 · openalex updated_date 2026/07/28

Abstract

Symmetries and equivariance are fundamental to the generalization of neural\nnetworks on domains such as images, graphs, and point clouds. Existing work has\nprimarily focused on a small number of groups, such as the translation,\nrotation, and permutation groups. In this work we provide a completely general\nalgorithm for solving for the equivariant layers of matrix groups. In addition\nto recovering solutions from other works as special cases, we construct\nmultilayer perceptrons equivariant to multiple groups that have never been\ntackled before, including \O(1,3), \O(5), \Sp(n),\nand the Rubik's cube group. Our approach outperforms non-equivariant baselines,\nwith applications to particle physics and dynamical systems. We release our\nsoftware library to enable researchers to construct equivariant layers for\narbitrary matrix groups.\n

Cited by

• Drawback of Enforcing Equivariance and its Compensation via the Lens of Expressive Power
• Partially Equivariant Reinforcement Learning in Symmetry-Breaking Environments
• A Differentiable Measure of Algebraic Complexity: Provably Exact Discovery of Group Structures
• Equivariant-Aware Structured Pruning for Efficient Edge Deployment: A Comprehensive Framework with Adaptive Fine-Tuning
• Learning with Category-Equivariant Representations for Human Activity Recognition
• Equivariant Neural Networks for General Linear Symmetries on Lie Algebras
• Matrix-Free Least Squares Solvers: Values, Gradients, and What to Do With Them
• Explicit Discovery of Nonlinear Symmetries from Dynamic Data
• FS-KAN: Permutation Equivariant Kolmogorov-Arnold Networks via Function Sharing
• Interpretable Discovery of One-parameter Subgroups: A Modular Framework for Elliptical, Hyperbolic, and Parabolic Symmetries
• Learning Equivariant Functions via Quadratic Forms
• CP²: Leveraging Geometry for Conformal Prediction via Canonicalization
• Understanding Learning Invariance in Deep Linear Networks
• GLGENN: A Novel Parameter-Light Equivariant Neural Networks Architecture Based on Clifford Geometric Algebras
• seq-JEPA: Autoregressive Predictive Learning of Invariant-Equivariant World Models
• A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models
• Governing Equation Discovery from Data Based on Differential Invariants
• RECON: Robust symmetry discovery via Explicit Canonical Orientation Normalization
• Discovering Symbolic Differential Equations with Symmetry Invariants

Related

• Universal Equivariant Multilayer Perceptrons
• Graph Automorphism Group Equivariant Neural Networks
• Brauer's Group Equivariant Neural Networks
• Clifford Group Equivariant Neural Networks
• How Jellyfish Characterise Alternating Group Equivariant Neural Networks