A family of spectral gradient methods for optimization

2018/12/07 by Yu‐Hong Dai, Yakui Huang, Dai, Yu-Hong +3 · 1 citation
Engineering · Mathematics · #90C20 #90C25 #90C30 #Advanced Optimization Algorithms Research #FOS: Mathematics #Iterative Methods for Nonlinear Equations #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques

paper · pdf · doi:10.48550/arxiv.1812.02974

openalex publication_date 2018/12/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose a family of spectral gradient methods, whose stepsize is determined by a convex combination of the long Barzilai-Borwein (BB) stepsize and the short BB stepsize. Each member of the family is shown to share certain quasi-Newton property in the sense of least squares. The family also includes some other gradient methods as its special cases. We prove that the family of methods is R-superlinearly convergent for two-dimensional strictly convex quadratics. Moreover, the family is R-linearly convergent in the any-dimensional case. Numerical results of the family with different settings are presented, which demonstrate that the proposed family is promising.

Citations

• Benchmarking optimization software with performance profiles

Cited by

• On Convergence and Stability of Two Extended BB-like Step Sizes

Related

• Gradient methods exploiting spectral properties
• On the asymptotic convergence and acceleration of gradient methods
• Analysis and Performance of the Barzilai-Borwein Step-Size Rules for Optimization Problems in Hilbert Spaces
• A gradient method exploiting the two dimensional quadratic termination property
• Equipping Barzilai-Borwein method with two dimensional quadratic termination property