Generalization of Quasi-Newton Methods: Application to Robust Symmetric Multisecant Updates

2020/11/06 by Damien Scieur, Lewis Liu, Scieur, Damien +5
Computer Science · Engineering · Mathematics · #Advanced Adaptive Filtering Techniques #Advanced Optimization Algorithms Research #FOS: Mathematics #Matrix Theory and Algorithms #Numerical Analysis (math.NA) #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques

paper · pdf · doi:10.48550/arxiv.2011.03358

openalex publication_date 2020/11/06 · openalex created_date 2022/10/01 · openalex updated_date 2026/07/28

Abstract

Quasi-Newton techniques approximate the Newton step by estimating the Hessian using the so-called secant equations. Some of these methods compute the Hessian using several secant equations but produce non-symmetric updates. Other quasi-Newton schemes, such as BFGS, enforce symmetry but cannot satisfy more than one secant equation. We propose a new type of quasi-Newton symmetric update using several secant equations in a least-squares sense. Our approach generalizes and unifies the design of quasi-Newton updates and satisfies provable robustness guarantees.

Related

• Advancing Multi-Secant Quasi-Newton Methods for General Convex Functions
• A Bregman Extension of quasi-Newton updates II: Convergence and Robustness Properties
• A modified BFGS quasi-Newton iterative formula
• Soft quasi-Newton: Guaranteed positive definiteness by relaxing the secant constraint
• A New Multipoint Symmetric Secant Method with a Dense Initial Matrix