A Limited Memory Algorithm for Bound Constrained Optimization

1995/09/01 by Richard H. Byrd, Peihuang Lu, Jorge Nocedal +1 · 205 citations
Computer Science · Mathematics · #Advanced Optimization Algorithms Research #Iterative Methods for Nonlinear Equations #Matrix Theory and Algorithms

paper · doi:10.1137/0916069

openalex publication_date 1995/09/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/03

Abstract

An algorithm for solving large nonlinear optimization problems with simple bounds is described. It is based on the gradient projection method and uses a limited memory BFGS matrix to approximate the Hessian of the objective function. It is shown how to take advantage of the form of the limited memory approximation to implement the algorithm efficiently. The results of numerical tests on a set of large problems are reported.

Citations

• On the limited memory BFGS method for large scale optimization
• Some numerical experiments with variable-storage quasi-Newton algorithms

Cited by

• Mono- and Polyauxic Growth Kinetics: A Semi-Mechanistic Framework for Complex Biological Dynamics
• Analytical Series Expansion for Efficient Gradient Evaluation in Multi-Qubit Optimal Control
• ShaplEIG: Bayesian Experimental Design for Shapley Value Estimation
• Enhanced predictions of measurable soil carbon pools in the Australian rangelands with the Millennial v2 model
• ARIMA_PLUS: Large-scale, Accurate, Automatic and Interpretable In-Database Time Series Forecasting and Anomaly Detection in Google BigQuery
• Parameter Estimation in Land Surface Models: Challenges and Opportunities With Data Assimilation and Machine Learning
• To MCMC or not to MCMC: Evaluating non-MCMC methods for Bayesian penalized regression
• Cardiocirculatory Computational Models for the Study of Hypertension
• Deducing Closed-Form Expressions for Bright-Solitons in Strongly Magnetized Plasmas with Physics Informed Symbolic Regression (PISR)
• Existence and numerical approximation of a one-dimensional Boussinesq system with variable coefficients on a finite interval
• ReSkmer: modeling repeats allows k-mer-based alignment-free methods to calculate population genomic distances
• Quantum feature-map learning with reduced resource overhead
• A Sensitivity-Based Method for Bilevel Optimization Problems: Theoretical Analysis and Computational Performance
• A Point Process Model of Skin Conductance Responses in a Stroop Task for Predicting Depression and Suicidal Ideation
• Suppressing leakage and maintaining robustness in transmon qubits: Signatures of a trade-off relation
• Study of nuclear magnetic resonance spectra with the multi-model multi-level quantum complex exponential least squares algorithm
• Learning to Solve Optimization Problems Constrained with Partial Differential Equations
• Economic Analysis of Dynamic Controlled Atmosphere Storage for Organic Apples
• Outlier Detection in Plantar Pressure: Human-Centered Comparison of Statistical Parametric Mapping and Explainable Machine Learning
• An inverse problem for linear system of dispersive equations
• GPU-accelerated FREDopt package for simultaneous dose and LETd proton radiotherapy plan optimization via superiorization methods
• High-fidelity inference of power grid frequency distributions
• SkillSmith: Learning to Compose Parametric Skills and Textual Knowledge
• Phylogenetic tree inference from single-cell RNA sequencing data with SCITE-RNA
• Optimization of centroidal Voronoi tessellations
• Fast Linear Solvers via AI-Tuned Markov Chain Monte Carlo-based Matrix Inversion
• Unit-Modified Weibull Distribution and Quantile Regression Model
• Demonstrating an unconditional separation between quantum and classical information resources
• Spatial disaggregation of time series
• Sample Efficient Certification of Discrete-Time Control Barrier Functions
• Phase-Sensitive Measurements on a Fermi-Hubbard Quantum Processor
• Learning from financial rewards and punishments reduces the in-group bias in social approach without changing the in-group bias in impressions
• Hybrid Quantum-Classical Simulations of Graphene Analogues: Adsorption Energetics Beyond DFT
• Convergence of Stochastic Gradient Methods for Wide Two-Layer Physics-Informed Neural Networks for the Poisson Equation
• Physics-informed coherent motions to predict Lagrangian trajectories
• Thermodynamically Consistent Hybrid and Permutation-Invariant Neural Yield Functions for Anisotropic Plasticity
• Constrained Optimization via Constraint-Induced Geometry: Implicit Feasible Dynamics and Optimality from Stationarity
• Numerical Analysis of Unsupervised Learning Approaches for Parameter Identification in PDEs
• MultiNMRFit: a software to fit 1D and pseudo-2D NMR spectra
• Holistic Bioprocess Development Across Scales Using Multi-Fidelity Batch Bayesian Optimization
• A learning-driven automatic planning framework for proton PBS treatments of H&N cancers
• Extended parameter shift rules with minimal derivative variance for parameterized quantum circuits
• Digital Quantum Simulation of Flat-Band and All-Bands-Flat Dynamics for Tunable Quantum Transport
• Projection-based multifidelity linear regression for data-scarce applications
• Comparison of variational quantum eigensolvers in light nuclei
• Constraint-Preserving Data Generation for Visuomotor Policy Learning
• Efficient dataset construction using active learning and uncertainty-aware neural networks for plasma turbulent transport surrogate models
• Swap Network Augmented Ansätze on Arbitrary Connectivity
• A-optimal Designs under Generalized Linear Models
• Enhancing Stability of Physics-Informed Neural Network Training Through Saddle-Point Reformulation
• Characterization of Lateral Amorphous Selenium Photodetectors for Low-Photon and VUV Detection at Cryogenic Temperatures
• A Hybrid Mixture Approach for Clustering and Characterizing Cancer Data
• Symmetry-based quantum algorithms for open-shop scheduling with hard constraints
• Causal Mechanism Estimation in Multi-Sensor Systems Across Multiple Domains
• Improving physics-informed neural network extrapolation via transfer learning and adaptive activation functions
• KP-PINNs: Kernel Packet Accelerated Physics Informed Neural Networks
• Evaluation of the transmission dynamics of Lawsonia intracellularis in swine
• VIP: Visual Information Protection through Adversarial Attacks on Vision-Language Models
• Prevention of Overfitting on Mesh-Structured Data Regressions with a Modified Laplace Operator
• A Search for Millimeter-Bright Blazars as Astrophysical Neutrino Sources
• A trust-region framework for optimization using Hermite kernel surrogate models
• Symbolic identification of tensor equations in multidimensional physical fields
• Integrating nano- and micrometer-scale energy deposition models for mechanistic prediction of radiation-induced DNA damage and cell survival
• Online design of experiments by active learning for nonlinear system identification
• Long-Range Interacting Particles on a Helix: A Statistical and Correlation Analysis of Equilibrium Configurations
• Systematic improvement of the quantum approximate optimisation ansatz for combinatorial optimisation using quantum subspace expansion
• Robust Control of High-dimensional Quantum Systems against Coherent and Incoherent Errors
• A penalized least squares estimator for extreme-value mixture models
• Boosting Variational Inference
• GollumFit: An IceCube Open-Source Framework for Binned-Likelihood Neutrino Telescope Analyses
• Solving Multi-Coloring Combinatorial Optimization Problems Using Hybrid Quantum Algorithms
• Analytic Solutions for D-optimal Factorial Designs under Generalized Linear Models
• Unified theory resolves phenological paradoxes in biocollection data by modeling phenophase duration during Bayesian inference
• Learning Parametric Convex Functions
• Hierarchical Style-based Networks for Motion Synthesis
• Learning GPLVM with arbitrary kernels using the unscented transformation
• Convex Covariate Clustering for Classification
• Computing The Analytic Connectivity of A Uniform Hypergraph
• Comparison of global and local optimization methods for intensity-based 2D–3D registration
• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• Comparing accounts of formant normalization against US English listeners' vowel perception
• Physics-informed machine learning
• How I won the "Chess Ratings - Elo vs the Rest of the World" Competition
• Unleashing High-Quality Image Generation in Diffusion Sampling Using Second-Order Levenberg-Marquardt-Langevin
• GARLIC: GAussian Representation LearnIng for spaCe partitioning
• General Algorithmic Search
• Simplifying Bayesian Optimization Via In-Context Direct Optimum Sampling
• On-site surrogates for large-scale calibration
• An Exact Penalty Approach for Equality Constrained Optimization over a Convex Set
• COSMOS: A Data-Driven Probabilistic Time Series simulator for Chemical Plumes across Spatial Scales
• PrivATE: Differentially Private Confidence Intervals for Average Treatment Effects
• Optimal dynamic thermal plant control: A study and benchmark
• Tracing the evolutionary history of the CCR5delta32 deletion via ancient and modern genomes
• Elastic Processing of Analytical Query Workloads on IaaS Clouds
• Fitting and Analysis Technique for Inconsistent Nuclear Data
• The Operator Product Expansion for Radial Lattice Quantization of 3D ϕ⁴ Theory
• Screening Sinkhorn Algorithm for Regularized Optimal Transport
• Simultaneous Optimization of Efficiency and Degradation in Tunable HTL-Free Perovskite Solar Cells with MWCNT-Integrated Back Contact Using a Machine Learning-Derived Polynomial Regressor
• Improved Surrogate Modeling using Machine Learning for Industrial Civil Aircraft Aerodynamics
• Decomposition of Water Demand Patterns Using Skewed Gaussian Distributions for Behavioral Insights and Operational Planning
• Non-Local Phase Estimation with a Rydberg-Superconducting Qubit Hybrid
• Qiskit Machine Learning: an open-source library for quantum machine learning tasks at scale on quantum hardware and classical simulators
• A NuSTAR study of quasi-periodic oscillations from the ultraluminous X-ray sources in M82
• Fast calibration of the LIBOR Market Model with Stochastic Volatility based on analytical gradient
• The Entire Quantile Path of a Risk-Agnostic SVM Classifier
• Bayesian Wavelet Shrinkage of the Haar-Fisz Transformed Wavelet Periodogram
• ACRE: Abstract Causal REasoning Beyond Covariation
• Understanding Complexity in VideoQA via Visual Program Generation
• Estimation of quantile inequality curves and measures based on grouped data
• Estimation of the Parameters of Symmetric Stable ARMA and ARMA-GARCH Models
• Multilevel Optimization: Geometric Coarse Models and Convergence Analysis
• Quantile Propagation for Wasserstein-Approximate Gaussian Processes
• The death of the Job plot, transparency, open science and online tools, uncertainty estimation methods and other developments in supramolecular chemistry data analysis
• Inverse design of all‐dielectric metasurfaces with accidental bound states in the continuum
• Audiovisual spatial recalibration but not integration is shaped by early sensory experience
• Texture Mixer: A Network for Controllable Synthesis and Interpolation of Texture
• Outlier Detection Using a Novel method: Quantum Clustering
• A new paradigm for computing hydrodynamic forces on particles in Euler-Lagrange point-particle simulations
• OTI on FIRE: Testing the Efficacy of Orbital Torus Imaging to Recover the Galactic Potential
• Quantum solver for single-impurity Anderson models with particle-hole symmetry
• ACSC: Automatic Calibration for Non-repetitive Scanning Solid-State LiDAR and Camera Systems
• EW D-optimal Designs for Experiments with Mixed Factors
• The Optimised Theta Method
• Modular proximal optimization for multidimensional total-variation regularization
• Partially Specified Space Time Autoregressive Model with Artificial Neural Network
• A SVBRDF Modeling Pipeline using Pixel Clustering
• War of Words: The Competitive Dynamics of Legislative Processes
• War of Words II: Enriched Models of Law-Making Processes
• Probabilistic single-particle cryo-EM ab initio 3D reconstruction in SIMPLE
• Automatic calibration and simultaneous estimation of bottom friction and open boundary conditions in a tidal model
• Regression of ranked responses when raw responses are censored
• NOVA: Symbolic Regression Discovery of Interpretable Car-Following and Lane-Change Models with Driver Heterogeneity
• Onsite matrix elements of the tight-binding Hamiltonian of a strained crystal: Application to silicon, germanium, and their alloys
• Viscous regularization and r-adaptive remeshing for finite element analysis of lipid membrane mechanics
• Riemannian Geometry of Optimal Rebalancing in Dynamic Weight Automated Market Makers
• Escaping the Hydrolysis Trap: An Agentic Workflow for Inverse Design of Durable Photocatalytic Covalent Organic Frameworks
• Event-Driven Query Expansion
• Revisiting Chebyshev Polynomial and Anisotropic RBF Models for Tabular Regression
• Nonparametric estimation of stochastic differential equations with sparse Gaussian processes
• Preferred Women’s Waist-to-Hip Ratio Variation over the Last 2,500 Years
• Hybrid perturbation methods based on statistical time series models
• Reconstruction of Fine-Scale Auroral Dynamics
• IPC – Isoelectric Point Calculator
• Stochastic Volatility, Jumps, and Rates: A Unified Framework for Option Pricing and Term-Structure Simulation
• Automated Kernel Discovery Towards Understanding High-dimensional Bayesian Optimization
• DiffLiB: High-fidelity differentiable modeling of lithium-ion batteries and efficient gradient-based parameter identification
• Warm terrestrial planet with half the mass of Venus transiting a nearby star
• Prior-Guided Symbolic Regression: Towards Scientific Consistency in Equation Discovery
• Efficient quantum state preparation through seniority driven operator selection
• A stochastic model for the assessment of the transmission pathways of heart and skeleton muscle inflammation, pancreas disease and infectious salmon anaemia in marine fish farms in Norway
• Model Reconstruction for Moment-based Stochastic Chemical Kinetics
• A Bures-Wasserstein Formulation of Matrix Decomposition Structural Equation Modeling
• Genomic structure in Europeans dating back at least 36,200 years
• Geometry of nonlinear least squares with applications to sloppy models and optimization
• Using models to improve optimizers for variational quantum algorithms
• Alternating size field optimizing and parameterization domain CAD model remeshing
• Advancing CMA-ES with Learning-Based Cooperative Coevolution for Scalable Optimization
• Differentiating Through Dual Prices: End-to-End Policy Learning Under Capacity Constraints
• Multilayer network approach to modeling authorship influence on citation dynamics in physics journals
• Physics-Informed Deep Learning for Traffic State Estimation
• Augmented Outcome-weighted Learning for Optimal Treatment Regimes
• Detection of intensity bursts using Hawkes processes: An application to high-frequency financial data
• One qubit as a universal approximant
• Predicting battery end of life from solar off-grid system field data using machine learning
• High-order curvilinear hybrid mesh generation for CFD simulations
• A Computational Framework for Slang Generation
• Rich Polymorphic Behavior of Wigner Bilayers
• The frequency and infrared brightness of circumstellar discs at white dwarfs
• GPfit: AnRPackage for Fitting a Gaussian Process Model to Deterministic Simulator Outputs
• TWO TRANSITING LOW DENSITY SUB-SATURNS FROM K2
• Design and performance of the spin asymmetries of the nucleon experiment
• Data assimilation method for experimental and first-principles data: Finite-temperature magnetization of (Nd,Pr,La,Ce)2(Fe,Co,Ni)14B
• Storyboard: Optimizing Precomputed Summaries for Aggregation
• Efficient optimization of state preparation in quantum networks using quantum trajectories
• Optimized Rhombic Experimental Dynamic Checkerboard Designs to Elucidate Pharmacodynamic Drug Interactions of Antibiotics
• Encoding Mechano-Memories in Actin Networks
• A Bayesian Framework for the Analog Reconstruction of Kymographs From Fluorescence Microscopy Data
• Statistical learning of geometric characteristics of wireless networks
• New interpretable statistics for large-scale structure analysis and generation
• Differentiable graph neural network simulator for forward and inverse modeling of multi-layered slope system with multiple material properties
• Deep learning and face recognition: the state of the art
• Adaptive Approximations of Inclusions in a Semilinear Elliptic Problem Related to Cardiac Electrophysiology
• Detecting stable adsorbates of (1S)-camphor on Cu(111) with Bayesian optimization
• Oxygen lines in solar granulation
• Inverse design of discrete mechanical metamaterials
• Dynamical Imaging with Interferometry
• Apprentice for Event Generator Tuning
• BOPfox program for tight-binding and analytic bond-order potential calculations
• Black hole spin in X-ray binaries: giving uncertainties an f
• Physical properties of a fan-shaped jet backlit by an X9.3 flare
• Quantum variational optimization: The role of entanglement and problem hardness
• On the consistency of inversion-free parameter estimation for Gaussian random fields
• Modeling residential solar PV adoption in Ireland: An agent-based modeling approach
• The reachable set of single-mode quadratic Hamiltonians
• Reduced-rank spatio-temporal modeling of air pollution concentrations in the Multi-Ethnic Study of Atherosclerosis and Air Pollution
• Hard-material Adhesion: Which Scales of Roughness Matter?
• Analytic semi-device-independent entanglement quantification for bipartite quantum states
• Superfast Line Spectral Estimation
• Arbitrary linear transformations for photons in the frequency synthetic dimension
• Accurate Tracking of Arabidopsis Root Cortex Cell Nuclei in 3D Time-Lapse Microscopy Images Based on Genetic Algorithm
• Efficient identification of linear, parameter-varying, and nonlinear systems with noise models
• Deep Learning-Based Prediction of Key Performance Indicators for Electrical Machines
• Beyond Coordinates: Meta-Equivariance in Statistical Inference
• Inverse Design of Photonic Crystals through Automatic Differentiation
• X-ray Strain and Stress Tensor Tomography