Learning (Approximately) Equivariant Networks via Constrained Optimization

2025/05/19 by Andrei Manolache, Manolache, Andrei, Luiz F. O. Chamon +3 · 2 citations
Computer Science · #Artificial Intelligence (cs.AI) #Boosting (machine learning) #Equivariant map #FOS: Computer and information sciences #Face and Expression Recognition #Generalization #Homogeneous space #Homotopy #Leverage (statistics) #Machine Learning (cs.LG) #Machine Learning and Algorithms #Neural Networks and Applications #Optimization problem #Robustness (evolution)

paper · pdf · doi:10.48550/arxiv.2505.13631

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2025/05/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Equivariant neural networks are designed to respect symmetries through their architecture, boosting generalization and sample efficiency when those symmetries are present in the data distribution. Real-world data, however, often departs from perfect symmetry because of noise, structural variation, measurement bias, or other symmetry-breaking effects. Strictly equivariant models may struggle to fit the data, while unconstrained models lack a principled way to leverage partial symmetries. Even when the data is fully symmetric, enforcing equivariance can hurt training by limiting the model to a restricted region of the parameter space. Guided by homotopy principles, where an optimization problem is solved by gradually transforming a simpler problem into a complex one, we introduce Adaptive Constrained Equivariance (ACE), a constrained optimization approach that starts with a flexible, non-equivariant model and gradually reduces its deviation from equivariance. This gradual tightening smooths training early on and settles the model at a data-driven equilibrium, balancing between equivariance and non-equivariance. Across multiple architectures and tasks, our method consistently improves performance metrics, sample efficiency, and robustness to input perturbations compared with strictly equivariant models and heuristic equivariance relaxations.

Citations

• Relaxed Equivariance via Multitask Learning
• Improving Equivariant Model Training via Constraint Relaxation
• Higher-Rank Irreducible Cartesian Tensors for Equivariant Message Passing
• Equivariant Graph Neural Operator for Modeling 3D Dynamics
• A Unified Framework to Enforce, Discover, and Promote Symmetry in Machine Learning
• Discovering Symmetry Breaking in Physical Systems with Relaxed Group Convolution
• Continuation Path Learning for Homotopy Optimization
• Resilient Constrained Learning
• Approximation-Generalization Trade-offs under (Approximate) Group Equivariance
• The Lie Derivative for Measuring Learned Equivariance
• MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields
• Single Loop Gaussian Homotopy Method for Non-convex Optimization
• Approximately Equivariant Networks for Imperfectly Symmetric Dynamics
• Residual Pathway Priors for Soft Equivariance Constraints
• Geometric and Physical Quantities Improve E(3) Equivariant Message Passing
• Highly accurate protein structure prediction with AlphaFold
• GemNet: Universal Directional Graph Neural Networks for Molecules
• Vector Neurons: A General Framework for SO(3)-Equivariant Networks
• E(n) Equivariant Graph Neural Networks
• On the Universality of Rotation Equivariant Point Cloud Networks
• SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks
• Probably Approximately Correct Constrained Learning
• Resilient Control: Compromising to Adapt
• Counterfactual Programming for Optimal Control
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Equivariant Flows: sampling configurations for multi-body systems with symmetric energies
• Fast Graph Representation Learning with PyTorch Geometric
• How Powerful are Graph Neural Networks?
• Clebsch-Gordan Nets: a Fully Fourier Space Spherical Convolutional Neural Network
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• Spectral Normalization for Generative Adversarial Networks
• On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups
• Spherical CNNs
• Dynamic Graph CNN for Learning on Point Clouds
• Dynamic Graph CNN for Learning on Point Clouds
• Graph Attention Networks
• Neural Message Passing for Quantum Chemistry
• Equivariance Through Parameter-Sharing
• Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis
• Harmonic Networks: Deep Translation and Rotation Equivariance
• Semi-Supervised Classification with Graph Convolutional Networks
• Asymptotic convergence of constrained primal-dual dynamics

Cited by

• Recurrent Equivariant Constraint Modulation: Learning Per-Layer Symmetry Relaxation from Data
• AL-CoLe: Augmented Lagrangian for Constrained Learning

Related

• Improving Equivariant Model Training via Constraint Relaxation
• A Tale of Two Symmetries: Exploring the Loss Landscape of Equivariant Models
• Recurrent Equivariant Constraint Modulation: Learning Per-Layer Symmetry Relaxation from Data
• Relaxing Equivariance Constraints with Non-stationary Continuous Filters
• LieAugmenter: Equivariant Learning by Discovering Symmetries with Learnable Augmentations