Subdifferentiation with symmetry

2025/09/14 by Cédric Josz, Josz, Cédric · 1 citation
Chemistry · Mathematics · #FOS: Mathematics #Graph theory and applications #History and advancements in chemistry #Optimization and Control (math.OC)

paper · pdf · doi:10.48550/arxiv.2509.11422

openalex publication_date 2025/09/14 · openalex created_date 2025/10/12 · openalex updated_date 2026/07/28

Abstract

Given an objective function that is invariant under an action of a Lie group, we study how its subgradients relate to the orbits of the action. Our main finding is that they satisfy projection formulae analogous to those stemming from the Whitney and Verdier stratifications. If the function is definable in an o-minimal structure on the real field, then we also obtain an invariant variational stratification. On the application side, we derive a conservation law for subgradient dynamics under minimal assumptions. It can be used to detect instability in discrete subgradient dynamics.

Citations

• Proximal random reshuffling under local Lipschitz continuity
• Keep the Momentum: Conservation Laws beyond Euclidean Gradient Flows
• A real moment-HSOS hierarchy for complex polynomial optimization with real coefficients
• Abide by the Law and Follow the Flow: Conservation Laws for Gradient\n Flows
• Improving Convergence and Generalization Using Parameter Symmetries
• Sufficient conditions for instability of the subgradient method with constant step size
• Symmetries, flat minima, and the conserved quantities of gradient flow
• On the Symmetries of Deep Learning Models and their Internal Representations
• Symmetry Teleportation for Accelerated Optimization
• Active manifolds, stratifications, and convergence to local minima in nonsmooth optimization
• Stochastic Subgradient Descent Escapes Active Strict Saddles on Weakly Convex Functions
• Noether: The More Things Change, the More Stay the Same
• Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning\n Dynamics
• Neural Teleportation
• Conservative set valued fields, automatic differentiation, stochastic gradient method and deep learning
• Exploring Weight Symmetry in Deep Neural Networks
• Algorithmic Regularization in Learning Deep Homogeneous Models: Layers\n are Automatically Balanced
• On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization
• Lasserre hierarchy for large scale polynomial optimization in real and complex variables
• Generic minimizing behavior in semi-algebraic optimization
• Curves of Descent
• Fixed-rank matrix factorizations and Riemannian low-rank optimization
• A rapidly-converging lower bound for the joint spectral radius via multiplicative ergodic theory
• Clarke subgradients of stratifiable functions
• Geometric categories and o-minimal structures
• Parameter Symmetry and Noise Equilibrium of Stochastic Gradient Descent

Cited by

• Reachability of gradient dynamics

Related

• On the stratification by orbit types II
• Stratified Subcartesian Spaces
• Reduction by invariants, stratifications, foliations, fibrations and relative equilibria, a short survey
• Differential invariants of the motion group actions
• Quasiconvexity versus group invariance