AL-CoLe: Augmented Lagrangian for Constrained Learning

2025/10/23 by Boero, Ignacio, Hounie, Ignacio, Ribeiro, Alejandro
#FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Signal Processing (eess.SP) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2510.20995

Abstract

Despite the non-convexity of most modern machine learning parameterizations, Lagrangian duality has become a popular tool for addressing constrained learning problems. We revisit Augmented Lagrangian methods, which aim to mitigate the duality gap in non-convex settings while requiring only minimal modifications, and have remained comparably unexplored in constrained learning settings. We establish strong duality results under mild conditions, prove convergence of dual ascent algorithms to feasible and optimal primal solutions, and provide PAC-style generalization guarantees. Finally, we demonstrate its effectiveness on fairness constrained classification tasks.

Citations

• Conditionally adaptive augmented Lagrangian method for physics-informed learning of forward and inverse problems
• SAFE: Finding Sparse and Flat Minima to Improve Pruning
• Constrained Sliced Wasserstein Embedding
• Learning (Approximately) Equivariant Networks via Constrained Optimization
• Simultaneous Multi-Robot Motion Planning with Projected Diffusion Models
• Near-Optimal Solutions of Constrained Learning Problems
• Learning Constrained Optimization with Deep Augmented Lagrangian Methods
• Strong Duality Relations in Nonconvex Risk-Constrained Learning
• Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraints
• A Lagrangian Duality Approach to Active Learning
• Probably Approximately Correct Constrained Learning
• Counterfactual Fairness

Related

• Near-Optimal Solutions of Constrained Learning Problems
• Learning Constrained Optimization with Deep Augmented Lagrangian Methods
• Lagrangian Duality for Constrained Deep Learning
• A new insight on augmented Lagrangian method with applications in machine learning
• Learning Valid Dual Bounds in Constraint Programming: Boosted Lagrangian Decomposition with Self-Supervised Learning