A Proximal-Gradient Method for Solving Regularized Optimization Problems with General Constraints

2025/12/29 by Curtis, Frank E., Qu, Xiaoyi, Robinson, Daniel P.
#49M37 #65K05 #65K10 #65Y20 #68Q25 #90C30 #90C60 #FOS: Mathematics #Optimization and Control (math.OC)

paper · doi:10.48550/arxiv.2512.23166

Abstract

We propose, analyze, and test a proximal-gradient method for solving regularized optimization problems with general constraints. The method employs a decomposition strategy to compute trial steps and uses a merit function to determine step acceptance or rejection. Under various assumptions, we establish a worst-case iteration complexity result, prove that limit points are first-order KKT points, and show that manifold identification and active-set identification properties hold. Preliminary numerical experiments on a subset of the CUTEst test problems and sparse canonical correlation analysis problems demonstrate the promising performance of our approach.

Citations

• Complexity of Linearized Perturbed Augmented Lagrangian Methods for Nonsmooth Nonconvex Optimization with Nonlinear Equality Constraints
• Sparse Array Beamformer Design via ADMM
• Level Constrained First Order Methods for Function Constrained Optimization
• Sparsity in Deep Learning: Pruning and growth for efficient inference\n and training in neural networks
• Rate-improved Inexact Augmented Lagrangian Method for Constrained Nonconvex Optimization
• SL1M: Sparse L1-norm Minimization for contact planning on uneven terrain
• On the Global Geometry of Sphere-Constrained Sparse Blind Deconvolution
• Inexact Successive Quadratic Approximation for Regularized Optimization
• Complexity of a quadratic penalty accelerated inexact proximal point method for solving linearly constrained nonconvex composite programs
• Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition
• Structured Nonconvex and Nonsmooth Optimization: Algorithms and Iteration Complexity Analysis
• A Reduced-Space Algorithm for Minimizing ℓ₁-Regularized Convex Functions
• Learning both Weights and Connections for Efficient Neural Networks
• Sparsity-Promoting Optimal Wide-Area Control of Power Networks
• On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming
• Benchmarking optimization software with performance profiles

Related

• A Proximal-Gradient Method for Constrained Optimization
• On the complexity of proximal gradient and proximal Newton-CG methods for \(ℓ₁\)-regularized Optimization
• Gradient Descent Methods for Regularized Optimization
• Inexact Proximal-Gradient Methods with Support Identification
• Inexact Proximal Gradient Methods for Non-convex and Non-smooth Optimization