An Inertial Block Proximal Linearized Method with Adaptive Momentum for Nonconvex and Nonsmooth Optimization

2026/08/06 by Weifeng Yang
Mathematics · Computer Science · #math.OC #cs.LG

paper · pdf

arxiv created 2026/08/06 · arxiv updated 2026/08/07

Abstract

In this paper, we consider a class of multiblock nonconvex nonsmooth optimization problems, which covers many applications such as the analysis of pre-earthquake anomalies and machine learning. To solve this class of problems, we propose the inertial block proximal linearized method with two-phase adaptive momentum (IBPL⁺-TP). Compared to the current methods, our method possesses three main advantages: (1) it introduces a two-phase adaptive momentum strategy to effectively update the extrapolation parameters, (2) it allows using two different extrapolation points to accelerate the convergence, (3) it allows the extrapolation parameters of these two extrapolation points to be independent of and unconstrained by all other parameters. While maintaining the above advantages, we prove that our method ensures the monotonic convergence of the objective function of this class of problems, and we also prove that the sequence generated by our method globally converges to a critical point, as well as establish the convergence rate of our method. To demonstrate the effectiveness of our method, we apply it to solve two nonconvex and nonsmooth machine learning problems, namely sparse nonnegative matrix factorization with ℓ₀-constraints and sparse nonnegative CP decomposition with ℓ₀-constraints. The numerical experimental results on solving these problems show that our method outperforms several state-of-the-art methods.

Citations

• Inertial Accelerated Stochastic Mirror Descent for Large-Scale Generalized Tensor CP Decomposition
• An Accelerated Block Proximal Framework with Adaptive Momentum for Nonconvex and Nonsmooth Optimization
• Adam-family Methods for Nonsmooth Optimization with Convergence Guarantees
• MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis
• Proximal Gradient Method with Extrapolation and Line Search for a Class of Nonconvex and Nonsmooth Problems
• Proximal Gradient Method with Extrapolation and Line Search for a Class of Non-convex and Non-smooth Problems
• Convergence Analysis of Proximal Gradient with Momentum for Nonconvex Optimization
• Inertial Proximal Alternating Linearized Minimization (iPALM) for Nonconvex and Nonsmooth Problems
• Global Convergence of ADMM in Nonconvex Nonsmooth Optimization
• A globally convergent algorithm for nonconvex optimization based on block coordinate update
• Proximal alternating linearized minimization for nonconvex and nonsmooth problems
• Alternating proximal gradient method for sparse nonnegative Tucker decomposition
• Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods
• The university of Florida sparse matrix collection
• Proximal alternating minimization and projection methods for nonconvex problems. An approach based on the Kurdyka-Lojasiewicz inequality
• Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality