A family of variable-metric methods derived by variational means

1970/01/01 by Donald Goldfarb · 72 citations
Mathematics · #Fractional Differential Equations Solutions #Iterative Methods for Nonlinear Equations #Advanced Optimization Algorithms Research

paper · doi:10.1090/s0025-5718-1970-0258249-6

Abstract

A new rank-two variable-metric method is derived using Greenstadt’s variational approach [ <italic>Math. Comp.</italic> , this issue]. Like the Davidon-Fletcher-Powell (DFP) variable-metric method, the new method preserves the positive-definiteness of the approximating matrix. Together with Greenstadt’s method, the new method gives rise to a one-parameter family of variable-metric methods that includes the DFP and rank-one methods as special cases. It is equivalent to Broyden’s one-parameter family [ <italic>Math. Comp.</italic> , v. 21, 1967, pp. 368–381]. Choices for the inverse of the weighting matrix in the variational approach are given that lead to the derivation of the DFP and rank-one methods directly.

Cited by

• Conditioning of quasi-Newton methods for function minimization
• Optimization Methods for Large-Scale Machine Learning
• Accurate Estimation of Diffusion Coefficients and their Uncertainties from Computer Simulation
• Bayesian Workflow for Generative Modeling in Computational Psychiatry
• Non-equilibrium Effects in Vibrational Modes Pumped by Inelastic Tunneling
• Statistical modelling of networked evolutionary public goods games
• Fast Linear Convergence of Randomized BFGS
• Targeted Maximum Likelihood Estimation using Exponential Families
• Efficient Semiparametric Estimation of Short-term and Long-term Hazard Ratios with Right-Censored Data
• Revealing the Partially Coherent Nature of Transport in IGZO
• Atomic forces from correlation energy functionals based on the adiabatic-connection fluctuation-dissipation theorem
• On-demand phase-field modeling: Three-dimensional Landau energy for HfO2 through machine learning
• Symmetry Dilemmas in Quantum Computing for Chemistry: A Comprehensive Analysis
• Evaluation of Foundational Machine Learned Interatomic Potentials for Migration Barrier Predictions
• Novel phases in the Fe-Si-O system at terapascal pressures
• Deep learning in neural networks: An overview
• Constraining dark matter halo profiles with symbolic regression
• Excited state preparation on a quantum computer through adiabatic light-matter coupling
• Edge-Dependent Superconductivity in Twisted Bismuth Bilayers
• nlKrylov: A Unified Framework for Nonlinear GCR-type Krylov Subspace Methods
• Closed-loop calculations of electronic structure on a quantum processor and a classical supercomputer at full scale
• Two-Component Mixture Model in the Presence of Covariates
• Explicit Superlinear Convergence Rates of Broyden's Methods in Nonlinear Equations
• Fast Screening of Minimum Energy Crossing Points with Semiempirical Tight-Binding Methods
• Giant Bulk-Rashba Splitting in Polar Topological Insulator BiSbTeSe₂
• Action constrained quasi-Newton methods
• High-Dimensional Gaussian Process Inference with Derivatives
• Fragmented superconductivity in the Hubbard model as solitons in Ginzburg–Landau theory
• Error-tolerant Multisecant Method for Nonlinearly Constrained\n Optimization
• BFGS-ADMM for Large-Scale Distributed Optimization
• User-Device Authentication in Mobile Banking using APHEN for Paratuck2\n Tensor Decomposition
• Binding Energies of Interstellar Complex Organic Molecules on Water Ice Surfaces: A Quantum Chemical Investigation
• Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods
• Determination of proton PDF uncertainties with Markov Chain Monte Carlo
• Functional Accelerated Failure Time Models for Predicting Time Since Cannabis Use
• An Interval Hessian-based line-search method for unconstrained nonconvex optimization
• Censored Regression for Modelling International Small Arms Trading and its "Forensic" Use for Exploring Unreported Trades
• A Practical Adaptive Subgame Perfect Gradient Method
• A Unified Perspective on Optimization in Machine Learning and Neuroscience: From Gradient Descent to Neural Adaptation
• A Theoretical Examination of Various Complexes of a Proposed Novel Chemosensor Material—Graphene/SiC
• Temporal Functional Factor Analysis of Brain Connectivity
• Operator Commutativity Screening and Progressive Operator Block Reordering toward Many-body Inspired Quantum State Preparation
• Many-body post-processing of density functional calculations using the variational quantum eigensolver for Bader charge analysis
• Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods
• Polynomial-Chaos-based Kriging
• Explicit Global Convergence Rates of BFGS without Line Search
• Regularized Overestimated Newton
• A regret minimization approach to fixed-point iterations
• Cluster Gauss-Newton method for finding multiple approximate minimisers of nonlinear least squares problems with applications to parameter estimation of pharmacokinetic models
• Machine-Learned Leftmost Hessian Eigenvectors for Robust Transition State Finding
• Mice alternate between discrete strategies during perceptual decision-making
• Practical Quasi-Newton Methods for Training Deep Neural Networks
• Quasi-Newton Methods: A New Direction
• Pathfinder: Parallel quasi-Newton variational inference
• Development of Deep Learning Optimizers: Approaches, Concepts, and Update Rules
• Variable Metric Random Pursuit
• Slice-Wise Initial State Optimization to Improve Cost and Accuracy of the VQE on Lattice Models
• Distributionally Robust Game Theory
• Multi-Fidelity Gaussian Process based Empirical Potential Development for Si:H Nanowires
• Quasi-Newton Methods: Superlinear Convergence Without Line Searches for Self-Concordant Functions
• Explicit Superlinear Convergence Rates of The SR1 Algorithm
• Simple Stepsize for Quasi-Newton Methods with Global Convergence Guarantees
• Bayesian minimisation of energy consumption in turbulent pipe flow via unsteady driving
• Combining Nonlinear FETI-DP Methods and Quasi-Newton Methods using an SQP Approach
• Improving Quasi-Newton Methods via Image and Projection Operators
• DAve-QN: A Distributed Averaged Quasi-Newton Method with Local Superlinear Convergence Rate
• Geometric direct minimization for low-spin restricted open-shell Hartree-Fock
• Preconditioned Discrete-HAMS: A Second-order Irreversible Discrete Sampler
• Stochastic Block BFGS: Squeezing More Curvature out of Data
• Discovering New Runge-Kutta Methods Using Unstructured Numerical Search
• Probabilistic Line Searches for Stochastic Optimization
• Newton-Stein Method: An optimization method for GLMs via Stein's Lemma

Related

• A new approach to variable metric algorithms
• Been Down So Long it Looks Like Up to Me: A Unified Derivation of Conjugate Gradient and Variable Metric Minimization
• Variable metric algorithms driven by averaged operators
• Tseng's Algorithm with Extrapolation from the Past Endowed with Variable Metrics and Error Terms
• First-order and second-order variants of the gradient descent in a unified framework