Proximal Gradient Method for Nonsmooth Optimization over the Stiefel Manifold

2018/11/02 by Shixiang Chen, Chen, Shixiang, Shiqian Ma +6
Engineering · Mathematics · #Advanced Optimization Algorithms Research #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (stat.ML) #Numerical methods in inverse problems #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #math.OC #stat.ML

paper · pdf · doi:10.48550/arxiv.1811.00980

openalex publication_date 2018/11/02 · arxiv created 2019/05/10 · arxiv updated 2019/05/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We consider optimization problems over the Stiefel manifold whose objective function is the summation of a smooth function and a nonsmooth function. Existing methods for solving this kind of problems can be classified into three classes. Algorithms in the first class rely on information of the subgradients of the objective function and thus tend to converge slowly in practice. Algorithms in the second class are proximal point algorithms, which involve subproblems that can be as difficult as the original problem. Algorithms in the third class are based on operator-splitting techniques, but they usually lack rigorous convergence guarantees. In this paper, we propose a retraction-based proximal gradient method for solving this class of problems. We prove that the proposed method globally converges to a stationary point. Iteration complexity for obtaining an ε-stationary solution is also analyzed. Numerical results on solving sparse PCA and compressed modes problems are reported to demonstrate the advantages of the proposed method.

Citations

• Convergence of inexact descent methods for nonconvex optimization on Riemannian manifolds
• Nearly unbiased variable selection under minimax concave penalty
• Sparse Principal Component Analysis
• Splitting Algorithms for the Sum of Two Nonlinear Operators
• Proximal alternating linearized minimization for nonconvex and nonsmooth problems
• Analysis of a complex of statistical variables into principal components.
• Generalized power method for sparse principal component analysis
• Fast Image Recovery Using Variable Splitting and Constrained Optimization
• Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality
• Primal-Dual Optimization Algorithms over Riemannian Manifolds: an Iteration Complexity Analysis

Related

• Smoothing algorithms for nonsmooth and nonconvex minimization over the stiefel manifold
• An Inexact Riemannian Proximal Gradient Method
• Riemannian Proximal Gradient Methods (extended version)
• Proximal methods for structured nonsmooth optimization over Riemannian submanifolds
• A Constraint Dissolving Approach for Nonsmooth Optimization over the Stiefel Manifold