Conditioning of quasi-Newton methods for function minimization

1970/01/01 by D. F. Shanno, David F. Shanno · 79 citations
Mathematics · Computer Science · #Advanced Optimization Algorithms Research #Iterative Methods for Nonlinear Equations #Matrix Theory and Algorithms

paper · pdf · doi:10.1090/s0025-5718-1970-0274029-x

Abstract

Quasi-Newton methods accelerate the steepest-descent technique for function minimization by using computational history to generate a sequence of approximations to the inverse of the Hessian matrix. This paper presents a class of approximating matrices as a function of a scalar parameter. The problem of optimal conditioning of these matrices under an appropriate norm as a function of the scalar parameter is investigated. A set of computational results verifies the superiority of the new methods arising from conditioning considerations to known methods.

Citations

• A family of variable-metric methods derived by variational means

Cited by

• Compensation of 1/1 and 2/2 error field in Wendelstein 7-X via divertor heat load symmetrization
• Optimization Methods for Large-Scale Machine Learning
• Accurate Estimation of Diffusion Coefficients and their Uncertainties from Computer Simulation
• Bayesian Workflow for Generative Modeling in Computational Psychiatry
• Non-equilibrium Effects in Vibrational Modes Pumped by Inelastic Tunneling
• Symmetric Rank-k Methods
• Statistical modelling of networked evolutionary public goods games
• First-principles study of the electronic and optical properties of Nb-doped MoSe₂ by tensile strain
• Fast Linear Convergence of Randomized BFGS
• Targeted Maximum Likelihood Estimation using Exponential Families
• Efficient Semiparametric Estimation of Short-term and Long-term Hazard Ratios with Right-Censored Data
• Revealing the Partially Coherent Nature of Transport in IGZO
• Adhesion Energy of Phosphorene on Different Pristine and Oxidized Metallic Substrates
• Penalized Fair Regression for Multiple Groups in Chronic Kidney Disease
• Atomic forces from correlation energy functionals based on the adiabatic-connection fluctuation-dissipation theorem
• On-demand phase-field modeling: Three-dimensional Landau energy for HfO2 through machine learning
• Symmetry Dilemmas in Quantum Computing for Chemistry: A Comprehensive Analysis
• Evaluation of Foundational Machine Learned Interatomic Potentials for Migration Barrier Predictions
• An hybrid stochastic Newton algorithm for logistic regression
• Deep learning in neural networks: An overview
• Constraining dark matter halo profiles with symbolic regression
• nlKrylov: A Unified Framework for Nonlinear GCR-type Krylov Subspace Methods
• Closed-loop calculations of electronic structure on a quantum processor and a classical supercomputer at full scale
• Single-Photon-Level Atomic Frequency Comb Storage in Room Temperature Alkali Vapour
• Two-Component Mixture Model in the Presence of Covariates
• Explicit Superlinear Convergence Rates of Broyden's Methods in Nonlinear Equations
• Fast Screening of Minimum Energy Crossing Points with Semiempirical Tight-Binding Methods
• Giant Bulk-Rashba Splitting in Polar Topological Insulator BiSbTeSe₂
• Action constrained quasi-Newton methods
• High-Dimensional Gaussian Process Inference with Derivatives
• Fragmented superconductivity in the Hubbard model as solitons in Ginzburg–Landau theory
• Error-tolerant Multisecant Method for Nonlinearly Constrained\n Optimization
• User-Device Authentication in Mobile Banking using APHEN for Paratuck2\n Tensor Decomposition
• Binding Energies of Interstellar Complex Organic Molecules on Water Ice Surfaces: A Quantum Chemical Investigation
• Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods
• Determination of proton PDF uncertainties with Markov Chain Monte Carlo
• Functional Accelerated Failure Time Models for Predicting Time Since Cannabis Use
• An Interval Hessian-based line-search method for unconstrained nonconvex optimization
• Censored Regression for Modelling International Small Arms Trading and its "Forensic" Use for Exploring Unreported Trades
• A Practical Adaptive Subgame Perfect Gradient Method
• A Unified Perspective on Optimization in Machine Learning and Neuroscience: From Gradient Descent to Neural Adaptation
• A Theoretical Examination of Various Complexes of a Proposed Novel Chemosensor Material—Graphene/SiC
• Temporal Functional Factor Analysis of Brain Connectivity
• Operator Commutativity Screening and Progressive Operator Block Reordering toward Many-body Inspired Quantum State Preparation
• A Split-Client Approach to Second-Order Optimization
• Many-body post-processing of density functional calculations using the variational quantum eigensolver for Bader charge analysis
• Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods
• Polynomial-Chaos-based Kriging
• Explicit Global Convergence Rates of BFGS without Line Search
• Regularized Overestimated Newton
• A regret minimization approach to fixed-point iterations
• Cluster Gauss-Newton method for finding multiple approximate minimisers of nonlinear least squares problems with applications to parameter estimation of pharmacokinetic models
• Machine-Learned Leftmost Hessian Eigenvectors for Robust Transition State Finding
• Mice alternate between discrete strategies during perceptual decision-making
• Practical Quasi-Newton Methods for Training Deep Neural Networks
• Quasi-Newton Methods: A New Direction
• Pathfinder: Parallel quasi-Newton variational inference
• Development of Deep Learning Optimizers: Approaches, Concepts, and Update Rules
• Variable Metric Random Pursuit
• Regularity dependence of the rate of convergence of the learning curve for Gaussian process regression
• Slice-Wise Initial State Optimization to Improve Cost and Accuracy of the VQE on Lattice Models
• Distributionally Robust Game Theory
• Entropy Search for Information-Efficient Global Optimization
• Complexity-Optimized Sparse Bayesian Learning for Scalable Classification Tasks
• Prompt Pirates Need a Map: Stealing Seeds helps Stealing Prompts
• BULL-ODE: Bullwhip Learning with Neural ODEs and Universal Differential Equations under Stochastic Demand
• Multi-Fidelity Gaussian Process based Empirical Potential Development for Si:H Nanowires
• Quasi-Newton Methods: Superlinear Convergence Without Line Searches for Self-Concordant Functions
• Explicit Superlinear Convergence Rates of The SR1 Algorithm
• QUBO-based training for VQAs on Quantum Annealers
• Simple Stepsize for Quasi-Newton Methods with Global Convergence Guarantees
• Bayesian minimisation of energy consumption in turbulent pipe flow via unsteady driving
• Combining Nonlinear FETI-DP Methods and Quasi-Newton Methods using an SQP Approach
• Improving Quasi-Newton Methods via Image and Projection Operators
• MRpro - open PyTorch-based MR reconstruction and processing package
• Geometric direct minimization for low-spin restricted open-shell Hartree-Fock
• Stochastic Block BFGS: Squeezing More Curvature out of Data
• Discovering New Runge-Kutta Methods Using Unstructured Numerical Search
• Probabilistic Line Searches for Stochastic Optimization

Related

• Quasi-Newton method of Optimization is proved to be a steepest descent method under the ellipsoid norm
• Quasi-Newton Methods: A New Direction
• Structured Quasi-Newton Methods for Optimization with Orthogonality Constraints
• Practical Inexact Proximal Quasi-Newton Method with Global Complexity Analysis
• Advancing Multi-Secant Quasi-Newton Methods for General Convex Functions