A Framework of Constraint Preserving Update Schemes for Optimization on Stiefel Manifold

2013/01/02 by Bo Jiang, Yu‐Hong Dai, Jiang, Bo +1 · 3 citations
Computer Science · Engineering · Mathematics · #Advanced Optimization Algorithms Research #FOS: Mathematics #Matrix Theory and Algorithms #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques

paper · pdf · doi:10.48550/arxiv.1301.0172

openalex publication_date 2013/01/02 · openalex created_date 2019/06/27 · openalex updated_date 2026/07/28

Abstract

This paper considers optimization problems on the Stiefel manifold X^TX=I_p, where X∈ ℝ^{n × p} is the variable and I_p is the p-by-p identity matrix. A framework of constraint preserving update schemes is proposed by decomposing each feasible point into the range space of X and the null space of X^T. While this general framework can unify many existing schemes, a new update scheme with low complexity cost is also discovered. Then we study a feasible Barzilai-Borwein-like method under the new update scheme. The global convergence of the method is established with an adaptive nonmonotone line search. The numerical tests on the nearest low-rank correlation matrix problem, the Kohn-Sham total energy minimization and a specific problem from statistics demonstrate the efficiency of the new method. In particular, the new method performs remarkably well for the nearest low-rank correlation matrix problem in terms of speed and solution quality and is considerably competitive with the widely used SCF iteration for the Kohn-Sham total energy minimization.

Citations

• Sparse Principal Component Analysis
• A Generalized Solution of the Orthogonal Procrustes Problem
• Non-monotone trust-region algorithms for nonlinear optimization subject to convex constraints

Cited by

• Retraction-Free Optimization over the Stiefel Manifold for the LoRA Fine-Tuning
• Quantum measurement tomography with mini-batch stochastic gradient descent
• Gradient type optimization methods for electronic structure calculations

Related

• Optimization without Retraction on the Random Generalized Stiefel Manifold
• A Theory of the NEPv Approach for Optimization On the Stiefel Manifold
• Energy-adaptive Riemannian optimization on the Stiefel manifold
• A relaxation method for binary optimizations on constrained Stiefel manifold
• Randomized Submanifold Subgradient Method for Optimization over Stiefel Manifolds