Benchmarking optimization software with performance profiles

2002/01/01 by Elizabeth D. Dolan, Jorge J. Moré, Jorge J. Morè · 4,398 citations
Computer Science · Engineering · Mathematics · #Advanced Optimization Algorithms Research #Benchmarking #Computer science #Data mining #Engineering #Formal Methods in Verification #Metric (unit) #Operations management #Parallel Computing and Optimization Techniques #Performance metric #Programming language #Software #Software development #Software metric #Software quality

paper · doi:10.1007/s101070100263

published in Mathematical Programming 91(2), 201-213 (Springer Science+Business Media)

openalex publication_date 2002/01/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Cited by

• Augmented Lagrangian methods under the constant positive linear dependence constraint qualification
• On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming
• Global convergence of a modified Fletcher–Reeves conjugate gradient method with Armijo-type line search
• SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization
• Fully Convergent Projection-based Methods with Momentum under Nonconvex Geometric
• Global Optimality Characterizations and Algorithms for Minimizing Quartically-Regularized Third-Order Taylor Polynomials
• Condensed Interior-Point Methods for Scalable Nonlinear Programming on GPUs
• From a Scalar to a Matrix Setting for the Dai--Liao Parameter
• Projected Gradient Methods with Momentum
• A family of Hager–Zhang conjugate gradient methods for system of monotone nonlinear equations
• Reinforcement Learning via Self-Distillation
• A Proximal-Gradient Method for Solving Regularized Optimization Problems with General Constraints
• Avoiding local minima in multilayer network optimization by incremental training
• A Homogeneous Tensor Framework for High-Order Trust-Region and Spherical Polynomial Optimization
• Near Minimax Line Spectral Estimation
• A variation of Broyden Class methods using Householder adaptive transforms
• An accelerated first-order method with complexity analysis for solving cubic regularization subproblems
• Efficient globalization of heavy-ball type methods for unconstrained optimization based on curve searches
• Projection and contraction methods with double inertial steps for variational inclusion problems on Hilbert spaces
• Parallel Model-Based Derivative-Free Optimization via Rank-Two KKT Updates
• Low-Rank KKT Updates and a Parallel Flipping Mechanism for Model-Based Derivative-Free Optimization
• Improving Matrix Exponential for Generative AI Flows: A Taylor-Based Approach Beyond Paterson--Stockmeyer
• Fast and Exact Least Absolute Deviations Line Fitting via Piecewise Affine Lower-Bounding
• All Emulators are Wrong, Many are Useful, and Some are More Useful Than Others: A Reproducible Comparison of Computer Model Surrogates
• OpenSQP: A Reconfigurable Open-Source SQP Algorithm in Python for Nonlinear Optimization
• Algorithm 851
• A descent extension of the Polak–Ribière–Polyak conjugate gradient method
• Extended-Krylov-subspace methods for trust-region and norm-regularization subproblems
• Krylov Subspace Acceleration for First-Order Splitting Methods in Convex Quadratic Programming
• Atomic norm denoising with applications to line spectral estimation
• Sequential Quadratic Optimization for Nonlinear Equality Constrained Stochastic Optimization
• Structural and Solution Analysis for the Ordered Weber Problem under Spatial Uncertainty
• A Unifying Framework for Sparsity Constrained Optimization
• Parallel Maximum Clique Algorithms with Applications to Network Analysis and Storage
• Stochastic Subspace Descent
• Two globally convergent nonmonotone trust-region methods for unconstrained optimization
• Deep Reinforcement Learning at the Edge of the Statistical Precipice
• QPPAL: A two-phase proximal augmented Lagrangian method for high dimensional convex quadratic programming problems
• Atomic Norm Denoising With Applications to Line Spectral Estimation
• A New Error in Variables Model for Solving Positive Definite Linear\n System Using Orthogonal Matrix Decompositions
• A trust-region approach for computing Pareto fronts in multiobjective optimization
• Branch-and-cut-and-price algorithms for the Degree Constrained Minimum Spanning Tree Problem
• A Two-Term PRP-Based Descent Method
• A family of spectral gradient methods for optimization
• Hashing embeddings of optimal dimension, with applications to linear least squares
• AILS-II: An Adaptive Iterated Local Search Heuristic for the Large-scale Capacitated Vehicle Routing Problem
• Modeling and solving a real-world cutting stock problem in the marble industry via mathematical programming and stochastic diffusion search approaches
• A one-phase interior point method for nonconvex optimization
• Parallel Surrogate-assisted Optimization Using Mesh Adaptive Direct Search
• Nonnegative Factorization and The Maximum Edge Biclique Problem
• AI Programmer: Autonomously Creating Software Programs Using Genetic Algorithms
• A hybrid combinatorial-continuous strategy for solving molecular distance geometry problems
• Quadratic Convergence of a Projection Method for a Plane Curve Feasibility Problem
• Self-Concordant Analysis of Frank-Wolfe Algorithms
• Generalized Reduced Jacobian Method
• GPU-Accelerated Algorithms for Process Mapping
• SPHERE: Spherical partitioning for large-scale routing optimization
• Path-Following Gradient-Based Decomposition Algorithms For Separable Convex Optimization
• Adaptive Pruning for Increased Robustness and Reduced Computational Overhead in Gaussian Process Accelerated Saddle Point Searches
• A Note on Performance Profiles for Benchmarking Software
• Computing Linear Combinations of φ-Function Actions for Exponential Integrators
• An Accelerated Newton-GMRES Method for Multilinear PageRank
• Gradient methods exploiting spectral properties
• ML2B: Benchmarking LLMs on Cross-Lingual ML Pipeline Generation
• A Riemannian AdaGrad-Norm Method
• A line search framework with restarting for noisy optimization problems
• Neural Kernels Without Tangents
• SeaPearl: A Constraint Programming Solver guided by Reinforcement Learning
• Sequential test sampling for stochastic derivative-free optimization
• Nonmonotone Trust-Region Methods for Optimization of Set-Valued Mapping of Finite Cardinality
• A novel hybrid trust region algorithm based on nonmonotone and LOOCV techniques
• Time-Fair Benchmarking for Metaheuristics: A Restart-Fair Protocol for Fixed-Time Comparisons
• Model-Driven Subspaces for Large-Scale Optimization with Local Approximation Strategy
• OSQP: An Operator Splitting Solver for Quadratic Programs
• Trust-Region Method for Optimization of Set-Valued Maps Given by Finitely Many Functions
• Alternating cyclic extrapolation methods for optimization algorithms
• Sobolev Seminorm of Quadratic Functions with Applications to Derivative-Free Optimization
• Stabilized Barzilai-Borwein method
• A Nonmonotone Front Descent Method for Bound-Constrained Multi-Objective Optimization
• Fast and Accurate SVD-Type Updating in Streaming Data
• Symmetry-breaking symmetry in directed spectral partitioning
• Trust-region Filter Algorithms utilising Hessian Information for Grey-Box Optimisation
• Complete results for a numerical evaluation of interior point solvers for large-scale optimal power flow problems
• Spectral conjugate gradient methods for vector optimization problems
• A nonlinear conjugate gradient algorithm for multiobjective optimization: multiple hybrid search direction and global rates
• Accelerated, Optimal, and Parallel: Some Results on Model-Based Stochastic Optimization
• An Efficient Hybridization of Graph Representation Learning and Metaheuristics for the Constrained Incremental Graph Drawing Problem
• Active-set Newton-MR methods for nonconvex optimization problems with bound constraints
• A Partially Derivative-Free Proximal Method for Composite Multiobjective Optimization in the Hölder Setting
• NLOptControl: A modeling language for solving optimal control problems
• An algorithmic framework for the optimization of deep neural networks architectures and hyperparameters
• L-SR1: Learned Symmetric-Rank-One Preconditioning
• How much progress have we made in neural network training? A New Evaluation Protocol for Benchmarking Optimizers
• Fast Convergence of Multiobjective Inertial Gradient Systems with Time Scaling
• Stadium norm and Douglas-Rachford splitting: a new approach to road design optimization
• Feature Selection in Nonlinear SVMs via Local Search and Submodular Optimization
• Accelerated derivative-free spectral residual method for nonlinear systems of equations
• Accelerating Quadratic Optimization with Reinforcement Learning
• The Augmented Mixing Method: Computing High-Accuracy Primal-Dual Solutions to Large-Scale SDPs via Column Updates
• ApproxJoin: Approximate Matching for Efficient Verification in Fuzzy Set Similarity Join
• On the global convergence of the inexact semi-smooth Newton method for absolute value equation
• A New Diagonal Quasi-Newton Updating Method With Scaled Forward Finite Differences Directional Derivative for Unconstrained Optimization
• Inexact Levenberg-Marquardt methods under Hölder metric subregularity
• Optimizing Damper Connectors for Adjacent Buildings
• Escaping Poor Local Minima in Large Scale Robust Estimation
• High Performance Block Incomplete LU Factorization
• Using a New Nonlinear Gradient Method for Solving Large Scale Convex Optimization Problems with an Application on Arabic Medical Text
• Vilin: Unconstrained Numerical Optimization Application
• A Semi-smooth Newton Method for Solving Semidefinite Programs in Electronic Structure Calculations
• A Visual Web Tool to Perform What-If Analysis of Optimization Approaches
• A thorough study of Riemannian Newton's Method
• Hybrid Riemannian Conjugate Gradient Methods with Global Convergence Properties
• Node Embedding via Word Embedding for Network Community Discovery
• A derivative-free Levenberg-Marquardt method for sparse nonlinear least squares problems
• A Gradient Sampling method based on Ideal direction for solving nonsmooth nonconvex optimization problems: convergence analysis and numerical experiments
• On Symmetric Rectilinear Matrix Partitioning
• On the asymptotic behavior of a higher-order extrapolation primal-dual interior-point method for nonlinear programming
• Preconditioning of Active-Set Newton Methods for PDE-constrained Optimal Control Problems
• On the iterative solution of systems of the form A^T A x=A^Tb+c
• A Decomposition Augmented Lagrangian Method for Low-rank Semidefinite Programming
• On efficiency of nonmonotone Armijo-type line searches
• A natural SQP method with potentially infeasible subproblems for nonlinear optimization
• On circumcentered direct methods for monotone variational inequality problems
• Stochastic Bundle Adjustment for Efficient and Scalable 3D Reconstruction
• The importance of structure in incomplete factorization preconditioners
• A Graduated Filter Method for Large Scale Robust Estimation
• On Convergence and Stability of Two Extended BB-like Step Sizes
• Optimal subgradient algorithms with application to large-scale linear inverse problems
• Is Sparse Matrix Reordering Effective for Sparse Matrix-Vector Multiplication?
• On Inexact Accelerated Proximal Gradient Methods with Relative Error Rules
• MSS: MATLAB Software for L-BFGS Trust-Region Subproblems for Large-Scale\n Optimization
• LLM-First Search: Self-Guided Exploration of the Solution Space
• A Newton Augmented Lagrangian Method for Symmetric Cone Programming with Complexity Analysis
• A simplicial decomposition framework for large scale convex quadratic programming
• Limited-Memory BFGS with Displacement Aggregation
• An exact separation algorithm for unsplittable flow capacitated network design arc-set polyhedron
• A new nonmonotone adaptive trust region algorithm
• Equipping Barzilai-Borwein method with two dimensional quadratic termination property
• A Multistage Stochastic Programming Approach to the Dynamic and Stochastic VRPTW - Extended version
• Algorithms for the continuous nonlinear resource allocation problem---new implementations and numerical studies
• On the acceleration of the Barzilai-Borwein method
• A scaling and recovering algorithm for the matrix φ-functions
• How far away are truly hyperparameter-free learning algorithms?
• Algorithm XXX: SC-SR1: Matlab software for solving shape-changing L-SR1 trust-region subproblems
• Solving Linear Programs with Complementarity Constraints using Branch-and-Cut
• On the effects of scaling on the performance of Ipopt
• A Structured Tour of Optimization with Finite Differences
• q-Line Search Scheme for Optimization Problem
• Minimal-to-Maximal Conversion Search Is Not Output-Polynomial
• StoMADS: Stochastic blackbox optimization using probabilistic estimates
• A Riemannian Optimization Approach for Finding the Nearest Reversible Markov Chain
• Using a thousand optimization tasks to learn hyperparameter search strategies
• Customized Interior-Point Methods Solver for Embedded Real-Time Convex Optimization
• COCO: Performance Assessment
• Dominant Arm Identification with Mixing and Recycling Observed Samples
• BROUTE: a benchmark suite for the implementation of standard vehicle routing algorithms
• GPU-Accelerated SPOCK for Scenario-Based Risk-Averse Optimal Control Problems
• Sufficient Descent Riemannian Conjugate Gradient Method
• Solving graph coloring problems with the Douglas-Rachford algorithm
• The equivalence between doubly nonnegative relaxation and semidefinite relaxation for binary quadratic programming problems
• A three-term Polak-Ribière-Polyak conjugate gradient method for vector optimization
• Comparing the Finite-Time Performance of Simulation-Optimization Algorithms
• Asymmetry PRISM: A CPU/GPU Portfolio Optimization Engine for Deadline-Bounded Institutional Rebalancing
• Evaluating Rust for Sparse Matrix Kernels in Scientific Computing
• Factoring nonnegative matrices with linear programs
• A Newton-Type Active Set Method for Nonlinear Optimization with Polyhedral Constraints
• Gradient Method for Optimization on Riemannian Manifolds with Lower Bounded Curvature
• Solving Multivariate Polynomial Systems and Rectangular Multiparameter Eigenvalue Problems with MacaulayLab
• Near-Optimal Minimum Cuts in Hypergraphs at Scale
• Convex Recoloring of General Graphs: Formulations, Polyhedra, and Computational Experiments
• A comparison of algorithms for maximum entropy parameter estimation
• Tool Choice Matters: Evaluating edgeR vs. DESeq2 for Sensitivity, Robustness, and Cross-Study Performance
• Optimization over Trained Neural Networks: Going Large with Gradient-Based Algorithms
• A Two-Stage Active-Set Algorithm for Bound-Constrained Optimization
• Linear-Time Multilevel Graph Partitioning via Edge Sparsification
• Multifidelity Bayesian Optimization for Binomial Output
• A Frank-Wolfe Based Branch-and-Bound Algorithm for Mean-Risk Optimization
• A Simulated Annealing Algorithm for the Directed Steiner Tree Problem
• Analytic Continued Fractions for Regression: A Memetic Algorithm Approach
• CAPO: Cost-Aware Prompt Optimization
• Safe Squeezing for Antisparse Coding
• An improved sequential quadratic programming algorithm for solving general nonlinear programming problems
• COCO: a platform for comparing continuous optimizers in a black-box setting
• A scenario decomposition algorithm for strategic time window assignment vehicle routing problems
• Dynamic Non-diagonal Regularization in Interior Point Methods for Linear and Convex Quadratic Programming
• A generalized worst-case complexity analysis for non-monotone line searches
• An Algorithm to Solve Cardinality Constrained Quadratic Optimization Problem with an Application to the Best Subset Selection in Regression
• Direct-search methods for decentralized blackbox optimization
• Deterministic Parallel High-Quality Hypergraph Partitioning
• Fast Gradient Algorithm with Dry-like Friction and Nonmonotone Line Search for Nonconvex Optimization Problems
• Learning to Guide Random Search
• BioPreDyn-bench: a suite of benchmark problems for dynamic modelling in systems biology. [europepmc]
• A Limited-Memory BFGS Algorithm Based on a Trust-Region Quadratic Model for Large-Scale Nonlinear Equations. [europepmc]
• Two New PRP Conjugate Gradient Algorithms for Minimization Optimization Models. [europepmc]
• A new Newton-like method for solving nonlinear equations. [europepmc]
• The Modified HZ Conjugate Gradient Algorithm for Large-Scale Nonsmooth Optimization. [europepmc]
• A modified three-term PRP conjugate gradient algorithm for optimization models. [europepmc]
• On the efficiency of nature-inspired metaheuristics in expensive global optimization with limited budget. [europepmc]
• A conjugate gradient algorithm for large-scale unconstrained optimization problems and nonlinear equations. [europepmc]
• A new filter QP-free method for the nonlinear inequality constrained optimization problem. [europepmc]
• An Efficient Orthonormalization-Free Approach for Sparse Dictionary Learning and Dual Principal Component Pursuit. [europepmc]
• On three-term conjugate gradient method for optimization problems with applications on COVID-19 model and robotic motion control. [europepmc]
• A branch-and-Benders-cut algorithm for a bi-objective stochastic facility location problem. [europepmc]
• Solving the clustered traveling salesman problem via traveling salesman problem methods. [europepmc]
• The global convergence of spectral RMIL conjugate gradient method for unconstrained optimization with applications to robotic model and image recovery. [europepmc]
• The global convergence of some self-scaling conjugate gradient methods for monotone nonlinear equations with application to 3DOF arm robot model. [europepmc]
• An efficient gradient-based algorithm with descent direction for unconstrained optimization with applications to image restoration and robotic motion control. [europepmc]
• Benchmarking the operation of quantum heuristics and Ising machines: scoring parameter setting strategies on optimization applications. [europepmc]
• An efficient three-term conjugate gradient algorithm with restart strategy and image restoration. [europepmc]
• Global convergence in a hybrid conjugate gradient projection method for finding solutions of constrained nonlinear equations with applications. [europepmc]
• Transfer learning for solar irradiation prediction in Minas Gerais, Brazil. [europepmc]
• Modified iterative double step length model for solving nonlinear systems with application to motion control. [europepmc]
• Employing Multiple Nonuniform Discrete-Time Grids for Continuous Production Scheduling MILPs. [europepmc]
• Developing Interface Force Fields for Water and Oxygen on Pt, and Pt 3 Ni , Pt 3 Co Alloy Surfaces for Proton-Exchange Membrane Fuel Cell (PEMFC) Applications. [europepmc]
• Comparative Evaluation of Optical Alignment Algorithms for Integrated Probe Cards in Photonic Wafer Testing. [europepmc]
• A novel adaptive quasi-Newton-type update and its global convergence without Lipschitz condition for constrained system of nonlinear monotone equations. [europepmc]