Conditioning of quasi-Newton methods for function minimization
1970/01/01 by D. F. Shanno, David F. Shanno · 3,091 citations
Computer Science · Mathematics · #Advanced Optimization Algorithms Research #Applied mathematics #Function (biology) #Hessian matrix #Inverse #Iterative Methods for Nonlinear Equations #Mathematical optimization #Mathematics #Matrix Theory and Algorithms #Minification #Newton's method #Quasi-Newton method #Scalar (mathematics) #Sequence (biology)
paper · pdf · doi:10.1090/s0025-5718-1970-0274029-x
published in Mathematics of Computation 24(111), 647-656 (American Mathematical Society (AMS))
crossref issued 1970/01/01 · crossref published 1970/01/01 · crossref published-print 1970/01/01 · openalex publication_date 1970/01/01 · crossref created 2010/06/30 · openalex created_date 2025/10/10 · crossref deposited 2026/04/20 · openalex updated_date 2026/08/05 · crossref indexed 2026/08/08
Abstract
Quasi-Newton methods accelerate the steepest-descent technique for function minimization by using computational history to generate a sequence of approximations to the inverse of the Hessian matrix. This paper presents a class of approximating matrices as a function of a scalar parameter. The problem of optimal conditioning of these matrices under an appropriate norm as a function of the scalar parameter is investigated. A set of computational results verifies the superiority of the new methods arising from conditioning considerations to known methods.
Citations
Cited by
- Compensation of 1/1 and 2/2 error field in Wendelstein 7-X via divertor heat load symmetrization
- Accurate Estimation of Diffusion Coefficients and their Uncertainties from Computer Simulation
- Bayesian Workflow for Generative Modeling in Computational Psychiatry
- Non-equilibrium Effects in Vibrational Modes Pumped by Inelastic Tunneling
- Symmetric Rank-k Methods
- Statistical modelling of networked evolutionary public goods games
- First-principles study of the electronic and optical properties of Nb-doped MoSe2 by tensile strain
- Fast Linear Convergence of Randomized BFGS
- Targeted Maximum Likelihood Estimation using Exponential Families
- Efficient Semiparametric Estimation of Short-term and Long-term Hazard Ratios with Right-Censored Data
- Revealing the Partially Coherent Nature of Transport in IGZO
- Adhesion Energy of Phosphorene on Different Pristine and Oxidized Metallic Substrates
- Penalized Fair Regression for Multiple Groups in Chronic Kidney Disease
- Atomic forces from correlation energy functionals based on the adiabatic-connection fluctuation-dissipation theorem
- On-demand phase-field modeling: Three-dimensional Landau energy for HfO2 through machine learning
- Symmetry Dilemmas in Quantum Computing for Chemistry: A Comprehensive Analysis
- Evaluation of Foundational Machine Learned Interatomic Potentials for Migration Barrier Predictions
- An hybrid stochastic Newton algorithm for logistic regression
- Deep Learning in Neural Networks: An Overview
- Constraining dark matter halo profiles with symbolic regression
- nlKrylov: A Unified Framework for Nonlinear GCR-type Krylov Subspace Methods
- Closed-loop calculations of electronic structure on a quantum processor and a classical supercomputer at full scale
- Single-Photon-Level Atomic Frequency Comb Storage in Room Temperature Alkali Vapour
- Two-component Mixture Model in the Presence of Covariates
- Explicit Superlinear Convergence Rates of Broyden's Methods in Nonlinear Equations
- Fast Screening of Minimum Energy Crossing Points with Semiempirical Tight-Binding Methods
- Giant Bulk-Rashba Splitting in Polar Topological Insulator BiSbTeSe2
- Action constrained quasi-Newton methods
- High-Dimensional Gaussian Process Inference with Derivatives
- Fragmented superconductivity in the Hubbard model as solitons in Ginzburg–Landau theory
- Error-tolerant Multisecant Method for Nonlinearly Constrained Optimization
- User-Device Authentication in Mobile Banking using APHEN for Paratuck2\n Tensor Decomposition
- Binding Energies of Interstellar Complex Organic Molecules on Water Ice Surfaces: A Quantum Chemical Investigation
- Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods
- Determination of proton PDF uncertainties with Markov chain Monte Carlo
- Functional Accelerated Failure Time Models for Predicting Time Since Cannabis Use
- An Interval Hessian-based line-search method for unconstrained nonconvex optimization
- Censored Regression for Modelling International Small Arms Trading and its "Forensic" Use for Exploring Unreported Trades
- A Practical Adaptive Subgame Perfect Gradient Method
- A Unified Perspective on Optimization in Machine Learning and Neuroscience: From Gradient Descent to Neural Adaptation
- A Theoretical Examination of Various Complexes of a Proposed Novel Chemosensor Material—Graphene/SiC
- Temporal Functional Factor Analysis of Brain Connectivity
- Operator Commutativity Screening and Progressive Operator Block Reordering toward Many-body Inspired Quantum State Preparation
- A Split-Client Approach to Second-Order Optimization
- Many-body post-processing of density functional calculations using the variational quantum eigensolver for Bader charge analysis
- Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods
- Polynomial-Chaos-based Kriging
- Explicit Global Convergence Rates of BFGS without Line Search
- Regularized Overestimated Newton
- A regret minimization approach to fixed-point iterations
- Cluster Gauss-Newton method for finding multiple approximate minimisers of nonlinear least squares problems with applications to parameter estimation of pharmacokinetic models
- Machine-Learned Leftmost Hessian Eigenvectors for Robust Transition State Finding
- Mice alternate between discrete strategies during perceptual decision-making
- Practical Quasi-Newton Methods for Training Deep Neural Networks
- Quasi-Newton Methods: A New Direction
- Pathfinder: Parallel quasi-Newton variational inference
- Development of Deep Learning Optimizers: Approaches, Concepts, and Update Rules
- Variable Metric Random Pursuit
- Regularity dependence of the rate of convergence of the learning curve for Gaussian process regression
- Slice-Wise Initial State Optimization to Improve Cost and Accuracy of the VQE on Lattice Models
- Distributionally Robust Game Theory
- Entropy Search for Information-Efficient Global Optimization
- Complexity-Optimized Sparse Bayesian Learning for Scalable Classification Tasks
- Prompt Pirates Need a Map: Stealing Seeds helps Stealing Prompts
- BULL-ODE: Bullwhip Learning with Neural ODEs and Universal Differential Equations under Stochastic Demand
- Multi-Fidelity Gaussian Process based Empirical Potential Development for Si:H Nanowires
- Quasi-Newton Methods: Superlinear Convergence Without Line Searches for Self-Concordant Functions
- Explicit Superlinear Convergence Rates of The SR1 Algorithm
- QUBO-based training for VQAs on Quantum Annealers
- Simple Stepsize for Quasi-Newton Methods with Global Convergence Guarantees
- Bayesian minimisation of energy consumption in turbulent pipe flow via unsteady driving
- Combining Nonlinear FETI-DP Methods and Quasi-Newton Methods using an SQP Approach
- Modelling silicate mineral interfaces for carbon dioxide sequestration: structure and stability of orthoenstatite surfaces
- Introducing ‘miniRECgap’ R package for simple gap-filling of missing eddy covariance CO2 flux measurements with classic nonlinear environmental response functions via GUI-supported R-scripts (case-study: In-sample gap-filling with ‘miniRECgap’ vs. MDS and an optimised shallow ANN in a ‘challenging’ peatland ecosystem)
- Improving Quasi-Newton Methods via Image and Projection Operators
- LigParGen web server: an automatic OPLS-AA parameter generator for organic ligands
- MRpro - open PyTorch-based MR reconstruction and processing package
- Geometric direct minimization for low-spin restricted open-shell Hartree-Fock
- Stochastic Block BFGS: Squeezing More Curvature out of Data
- Discovering New Runge-Kutta Methods Using Unstructured Numerical Search
- Probabilistic Line Searches for Stochastic Optimization
- Newton-Stein Method: An optimization method for GLMs via Stein's Lemma
- Mechanistic insights into the dry oxidation of amorphous silicon nitride: a DFT study
- A New Diagonal Quasi-Newton Updating Method With Scaled Forward Finite Differences Directional Derivative for Unconstrained Optimization
- Chromatic Aberration Recovery on Arbitrary Images
- Limited-Memory Matrix Adaptation for Large Scale Black-box Optimization
- Greedy and Random Broyden's Methods with Explicit Superlinear Convergence Rates in Nonlinear Equations
- On the construction of probabilistic Newton-type algorithms
- Optimization Methods for Large-Scale Machine Learning
- Vilin: Unconstrained Numerical Optimization Application
- NysAct: A Scalable Preconditioned Gradient Descent using Nystrom Approximation
- Sketch and Project: Randomized Iterative Methods for Linear Systems and Inverting Matrices
- Efficient Stochastic BFGS methods Inspired by Bayesian Principles
- Enhanced Quantum behavior on frustrated Ising model: Quantum Approximate Optimization Algorithm study
- Explicit pseudo-transient continuation and the trust-region updating strategy for unconstrained optimization
- Distributed Second Order Methods with Fast Rates and Compressed Communication
- A trust-region framework for optimization using Hermite kernel surrogate models
- Affine-Invariant Global Non-Asymptotic Convergence Analysis of BFGS under Self-Concordance
- Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization
- Benchmarking of Oxygen Adsorption using TPD Spectroscopy for Accurate DFT Prediction of ORR on Anatase Titanium Dioxide (101)
- Defects at Play: Shaping the Photophysics and Photochemistry of Ice
- Explicit Convergence Rates of Greedy and Random Quasi-Newton Methods
- A Computationally Efficient Limited Memory CMA-ES for Large Scale Optimization
- In-Poor IGZO: Superior Resilience to Hydrogen in Forming Gas Anneal and PBTI
- An approximate Riemann solver approach in Physics-Informed Neural Networks for hyperbolic conservation laws
- Fast and Accurate Binary Response Mixed Model Analysis via Expectation Propagation
- Convex Latent Effect Logit Model via Sparse and Low-rank Decomposition
- QForte: an efficient state simulator and quantum algorithms library for molecular electronic structure
- Deterministic Optimisation of Jastrow Factors
- Do optimization methods in deep learning applications matter?
- Limited-Memory BFGS with Displacement Aggregation
- An Algorithm for Unconstrained Quadratically Penalized Convex Optimization
- Regional chemical potential analysis for material surfaces
- The Plateau Problem in the Heteroskedastic Probit Model
- A Self-Correcting Variable-Metric Algorithm Framework for Nonsmooth Optimization
- MACS: Multi-Agent Reinforcement Learning for Optimization of Crystal Structures
- Stable partial dislocation complexes in GaN as charge carrier lifetime modifiers for terahertz device applications by molecular dynamics and first-principle simulations
- Tensor Normal Training for Deep Learning Models
- Large scale digital prostate pathology image analysis combining feature extraction and deep neural network
- Overfitting has a limitation: a model-independent generalization gap bound based on Rényi entropy
- A New Training Method for Feedforward Neural Networks Based on Geometric Contraction Property of Activation Functions
- Incorporating Preconditioning into Accelerated Approaches: Theoretical Guarantees and Practical Improvement
- Computing planar and spherical choreographies
- Randomized Quasi-Newton Updates are Linearly Convergent Matrix Inversion Algorithms
- Modeling Nonstationary Time Series using Locally Stationary Basis Processes
- A Quasi-Newton Method to Solve Uncertain Multiobjective Optimization Problems with Uncertainty Set of Finite Cardinality
- Finding unstable periodic orbits: a hybrid approach with polynomial optimization
- MoreFit: A More Optimised, Rapid and Efficient Fit
- Worst case approach in convex minimization problems with uncertain data
- LITMUS: Bayesian Lag Recovery in Reverberation Mapping with Fast Differentiable Models
- Convergence rates of sub-sampled Newton methods
- Efficient Optimization of the Likelihood Function in Gaussian Process Modelling
- On low complexity Acceleration Techniques for Randomized Optimization:\n Supplementary Online Material
- FTNILO: Explicit Multivariate Function Inversion, Optimization and Counting, Cryptography Weakness and Riemann Hypothesis Solution Equation with Tensor Networks
- A penalized likelihood approach to estimate within-household contact networks from egocentric data
- DirichletReg: Dirichlet Regression for Compositional Data in R
- Computational Methods for the Analysis of Chemical Sensor Array Data from Volatile Analytes
- Software package MaRDI Open Interfaces for improved interoperability in numerical optimization
- MaRDI Open Interfaces for Interoperable Nonlinear Optimization
- From Persistent Homology to Reinforcement Learning with Applications for Retail Banking
- Hybrid perturbation methods based on statistical time series models
- The sequential megafaunal collapse hypothesis: Testing with existing data
- Pulse-optimised circuit elements for scalable and noise-resilient quantum chemistry
- Learning on-top: regressing the on-top pair density for real-space visualization of electron correlation
- Distributed Quantum-Enhanced Optimization: A Topographical Preconditioning Approach for High-Dimensional Search
- Rapid passage to ordered states in Rydberg atom arrays
- Coherence-based Approximate Derivatives via Web of Affine Spaces Optimization
- Probing the Bonding and Electronic Structure of Single Atom Dopants in Graphene with Electron Energy Loss Spectroscopy
- Reaching optimized parameter set: protein secondary structure prediction using neural network
- Forecasting with Multiple Seasonality
- A Practical Guide to Optimizing Industrial Thermal Spraying through Comparative Multi-Objective Optimization
- Semiparametric robust mixture of experts based on nonparametric maximum likelihood
- An Exchange-Correlation Functional for Fast and Accurate Modeling of Ferroelectric Perovskites
- Binding Energies of Interstellar Molecules on Crystalline and Amorphous Models of Water Ice by Ab Initio Calculations
- Preconditioning Natural and Second Order Gradient Descent in Quantum Optimization: A Performance Benchmark
- Adaptive cluster approximation for reduced density-matrix functional theory
- Quantum Approximate Optimization Algorithm: Performance, Mechanism, and Implementation on Near-Term Devices
- Avoiding local minima in variational quantum eigensolvers with the natural gradient optimizer
- Quantum variational optimization: The role of entanglement and problem hardness
- Gaussian Conversion Protocols for Cubic Phase State Generation
- COCO: a platform for comparing continuous optimizers in a black-box setting
- Optimizing quantum codes with an application to the loss channel with partial erasure information
- Accelerated Stochastic Matrix Inversion: General Theory and Speeding up BFGS Rules for Faster Second-Order Optimization
- Easy plane baby Skyrmions
- Strain and electronic properties at the van der Waals interface of phosphorus/boron nitride heterobilayers
- Thermal conductivity and Lorenz ratio of metals at intermediate temperatures with mode-level first-principles analysis
- Efficient Gradient-Enhanced Bayesian Optimizer with Comparisons to Conjugate-Gradient and Quasi-Newton Optimizers for Unconstrained Local Optimization
- Deriving the Gradients of Some Popular Optimal Transport Algorithms
- Representation Meets Optimization: Training PINNs and PIKANs for Gray-Box Discovery in Systems Pharmacology
- Violation of the third law of black hole mechanics in vacuum gravity
- Estimation of allele frequency and association mapping using next-generation sequencing data. [europepmc]
- Role of hydrogen in volatile behaviour of defects in SiO 2 -based electronic devices. [europepmc]
- The impact of temperature changes on vector-borne disease transmission: Culicoides midges and bluetongue virus. [europepmc]
- Focal Spot and Wavefront Sensing of an X-Ray Free Electron laser using Ronchi shearing interferometry. [europepmc]
- Diagnostic potential for a serum miRNA neural network for detection of ovarian cancer. [europepmc]
- Understanding the adsorption process in ZIF-8 using high pressure crystallography and computational modelling. [europepmc]
- Physiologically based pharmacokinetic-quantitative systems toxicology and safety (PBPK-QSTS) modeling approach applied to predict the variability of amitriptyline pharmacokinetics and cardiac safety in populations and in individuals. [europepmc]
- Substrate selectivity in starch polysaccharide monooxygenases. [europepmc]
- Catecholaminergic modulation of meta-learning. [europepmc]
- Restricted-Variance Molecular Geometry Optimization Based on Gradient-Enhanced Kriging. [europepmc]
- Restricted-Variance Constrained, Reaction Path, and Transition State Molecular Optimizations Using Gradient-Enhanced Kriging. [europepmc]
- Speech-Based Surgical Phase Recognition for Non-Intrusive Surgical Skills' Assessment in Educational Contexts. [europepmc]
- Optimizing Molecular Geometries in Strong Magnetic Fields. [europepmc]
- First-principles calculations of hybrid inorganic-organic interfaces: from state-of-the-art to best practice. [europepmc]
- Speed-of-sound imaging using diverging waves. [europepmc]
- Crystal face dependent intrinsic wettability of metal oxide surfaces. [europepmc]
- Gradient Expansions for the Large-Coupling Strength Limit of the Møller-Plesset Adiabatic Connection. [europepmc]
- Effect of the number of nitrogen dopants on the electronic and magnetic properties of graphitic and pyridinic N-doped graphene - a density-functional study. [europepmc]
- Interaction of carbohydrate binding module 20 with starch substrates. [europepmc]
- Chemoton 2.0: Autonomous Exploration of Chemical Reaction Networks. [europepmc]
- Particle fusion of super-resolution data reveals the unit structure of Nup96 in Nuclear Pore Complex. [europepmc]
- High-Quality Conformer Generation with CONFORGE: Algorithm and Performance Assessment. [europepmc]
- Stabilized and Controlled Release of Radicals within Copper Formate-Based Nanozymes for Biosensing. [europepmc]
- Modelling chemical processes in explicit solvents with machine learning potentials. [europepmc]
- The QCML dataset, Quantum chemistry reference data from 33.5M DFT and 14.7B semi-empirical calculations. [europepmc]
Related