Analysis of the BFGS Method with Errors

2019/01/25 by Yuchen Xie, Xie, Yuchen, Richard Byrd +3 · 4 citations
Computer Science · Engineering · Mathematics · #Advanced Optimization Algorithms Research #FOS: Mathematics #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.1901.09063

openalex publication_date 2019/01/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The classical convergence analysis of quasi-Newton methods assumes that the function and gradients employed at each iteration are exact. In this paper, we consider the case when there are (bounded) errors in both computations and establish conditions under which a slight modification of the BFGS algorithm with an Armijo-Wolfe line search converges to a neighborhood of the solution that is determined by the size of the errors. One of our results is an extension of the analysis presented in Byrd, R. H., & Nocedal, J. (1989), which establishes that, for strongly convex functions, a fraction of the BFGS iterates are good iterates. We present numerical results illustrating the performance of the new BFGS method in the presence of noise.

Citations

• Numerical optimization

Cited by

• Diffusion differentiable resampling
• Technical results on the convergence of quasi-Newton methods for nonsmooth optimization
• A line search framework with restarting for noisy optimization problems
• Active-Set Identification in Noisy and Stochastic Optimization

Related

• A Noise-Tolerant Quasi-Newton Algorithm for Unconstrained Optimization
• On the convergence of BFGS on a class of piecewise linear non-smooth functions
• Block BFGS Methods
• Affine-Invariant Global Non-Asymptotic Convergence Analysis of BFGS under Self-Concordance
• Sharpened Quasi-Newton Methods: Faster Superlinear Rate and Larger Local Convergence Neighborhood