An efficient method for finding the minimum of a function of several variables without calculating derivatives

1964/02/01 by M. J. D. Powell · 4,622 citations
Computer Science · Mathematics · #Advanced Optimization Algorithms Research #Algorithm #Applied mathematics #Computer science #Convergence (economics) #Function (biology) #Iterative Methods for Nonlinear Equations #Key (lock) #Mathematical optimization #Mathematics #Matrix Theory and Algorithms #Quadratic equation #Rate of convergence #Simple (philosophy) #Variation (astronomy)

paper · doi:10.1093/comjnl/7.2.155

published in The Computer Journal 7(2), 155-162 (Oxford University Press)

openalex publication_date 1964/02/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

A simple variation of the well-known method of minimizing a function of several variables by changing one parameter at a time is described. This variation is such that when the procedure is applied to a quadratic form, it causes conjugate directions to be chosen, so the ultimate rate of convergence is fast when the method is used to minimize a general function. A further variation completes the method, and its ensures that the convergence rate from a bad approximation to a minimum is always efficient. Practical applications of the procedure have proved to be very satisfactory, and numerical examples are given in which functions of up to twenty variables are minimized.

Cited by

• Molecular modelling with spherical Gaussians
• Gaussian Expansions of Atomic Orbitals
• Linear stability analysis of a pair plasma with arbitrary non-neutrality in the magnetic field of an infinite, straight wire
• Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods
• A quantitative assessment of dual energy computed tomography‐based material decomposition for imaging bone marrow edema associated with acute knee injury
• Hamiltonian expressibility for ansatz selection in variational quantum algorithms
• Large future genetic diversity losses are predicted even with habitat protection
• Large-scale acoustic peak shift in the Planck CMB data
• TuPAQ: An Efficient Planner for Large-scale Predictive Analytic Queries
• Gait transition in swimming
• Event Discovery in Time Series
• Experimental Comparison of Visual and Single-Receiver GPS Odometry
• Analytical Solutions for Turbulent Channel Flow Using Alexeev and Navier-Stokes Hydrodynamic Equations: Comparison with Experiments
• The Simons Observatory: forecasted constraints on primordial gravitational waves with the expanded array of Small Aperture Telescopes
• Structural Optimization in Tensor LEED Using a Parameter Tree and R-Factor Gradients
• Exposing Vulnerabilities in Counterfeit Prevention Systems Utilizing Physically Unclonable Surface Features
• Detection of Digital Facial Retouching utilizing Face Beauty Information
• Fast generation of 3D flow obstacles from parametric surface models: application to cardiac valves
• Selective recovery of critical materials in supercritical water desalination
• A Nonparametric Maximum Likelihood Approach to Mixture of Regression
• MRSimulator: A cross-platform, object-oriented software package for rapid solid-state NMR spectral simulation and analysis
• Fitting Reinforcement Learning Model to Behavioral Data under Bandits
• Universal derivative-free optimization method with quadratic convergence
• A new Lamarckian genetic algorithm for flexible ligand‐receptor docking
• apsis - Framework for Automated Optimization of Machine Learning Hyper Parameters
• Unlocking the Power of Orbital-Free Density Functional Theory to Explore the Electronic Structure Under Extreme Conditions
• A spectrum of tectonic processes at coronae on Venus revealed by gravity and topography
• SOIC: Semantic Online Initialization and Calibration for LiDAR and Camera
• The Method of Strained Coordinates for Vibrations with Weak Unilateral Springs
• A Unified Perspective on Optimization in Machine Learning and Neuroscience: From Gradient Descent to Neural Adaptation
• Statistical Benchmarking of Optimization Methods for Variational Quantum Eigensolver under Quantum Noise
• HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Video
• GPU-accelerated FREDopt package for simultaneous dose and LETd proton radiotherapy plan optimization via superiorization methods
• Insights into the 1968-1997 Dasht-e-Bayaz and Zirkuh earthquake sequences, eastern Iran, from calibrated relocations, InSAR and high-resolution satellite imagery
• Semi-Supervised Deep Metrics for Image Registration
• parSAT: Parallel Solving of Floating-Point Satisfiability
• Retarded kernels for longitudinal survival analysis and dynamic prediction
• Benchmarking Optimization Algorithms for Automated Calibration of Quantum Devices
• Data assimilation for energy-aware hybrid models
• Towards a Unified Framework for Pseudo-Nambu-Goldstone Dark Matter and Electroweak Baryogenesis
• 4D Virtual Imaging Platform for Dynamic Joint Assessment via Uni-Plane X-ray and 2D-3D Registration
• Freeze and Conquer: Reusable Ansatz for Solving the Traveling Salesman Problem
• Differentiable Forward and Back-Projector for Rigid Motion Estimation in X-ray Imaging
• Gaussian basis sets for all-electron excited-state calculations of large molecules
• Parity Cross-Resonance: A Multiqubit Gate
• Extended parameter shift rules with minimal derivative variance for parameterized quantum circuits
• KiDS-1000 Cosmology: Multi-probe weak gravitational lensing and spectroscopic galaxy clustering constraints
• Deep Information Theoretic Registration
• Model-based framework for automated quantification of error sources in quantum state tomography
• Explicit Gradient Learning
• Impact of model uncertainty on SPARC operating scenario predictions with empirical modeling
• Direction-Preserving Number Representations
• Chromatic Aberration Recovery on Arbitrary Images
• Robust and Scalable Techniques for TWR and TDoA based localization using Ultra Wide Band Radios
• Comparison of global and local optimization methods for intensity-based 2D–3D registration
• Computer-Assisted Analysis of Biomedical Images
• Applying quantum approximate optimization to the heterogeneous vehicle routing problem
• Statistical Inference for Polyak-Ruppert Averaged Zeroth-order Stochastic Gradient Algorithm
• Complete and Orthonormal Sets of Exponential-type Orbitals with non-integer quantum numbers. On the results for many-electron atoms using Roothaan's LCAO method
• Tuning without Peeking: Provable Generalization Bounds and Robust LLM Post-Training
• Quantum Machine Learning for State Tomography Using Classical Data
• Size-Dependent Tensile Behavior of Nanocrystalline HfNbTaTiZr High-Entropy Alloy: Roles of Solid-Solution and Short-Range Order
• Low-Order Modeling and High-Fidelity Simulations for the Prediction of Combustion Instabilities in Liquid Rocket Engines and Gas Turbines
• Enhancing Variational Quantum Algorithms for Multicriteria Optimization
• A Model-Based Derivative-Free Optimization Algorithm for Partially Separable Problems
• Engineering Precise and Robust Effective Hamiltonians
• The JHU-Microsoft Submission for WMT21 Quality Estimation Shared Task
• Parallelisation, initialisation, and boundary treatments for the diamond\n scheme
• Privacy-Preserving Generalized Linear Models using Distributed Block Coordinate Descent
• Deep Learning on Graphs: A Survey
• SciPy 1.0--Fundamental Algorithms for Scientific Computing in Python
• Balancing Profit and Fairness in Risk-Based Pricing Markets
• Hybrid-graph neural network method for muon fast reconstruction in neutrino telescopes
• VideoMarkBench: Benchmarking Robustness of Video Watermarking
• Problem specific ion native ansatz for combinatorial optimization
• Computational Modeling of Visual Texture Segregation
• Learning hidden cascades via classification
• Velocity variations at Columbia Glacier captured by particle filtering of oblique time-lapse images
• A Standardized Benchmark Set of Clustering Problem Instances for Comparing Black-Box Optimizers
• Least absolute deviations estimation for the censored regression model
• Maximization by Quadratic Hill-Climbing
• The Duck Redux: An Improved Proper‐Motion Upper Limit for the Pulsar B1757−24 near the Supernova Remnant G5.4–1.2
• Articulated Hands
• Simulation-based optimization of transportation costs in high pressure\n gas grid
• Chunking and redintegration in verbal short-term memory.
• LensPerfect: Gravitational Lens Mass Map Reconstructions Yielding Exact Reproduction of All Multiple Images
• Properties of the skyrmion crystal SkX-2 in the Heisenberg triangular lattice with scalar chirality
• First Interstellar Detection of Methyl Carbamate: A New Observational Anchor for Glycine Chemistry
• Computationally lightweight classifiers with frequentist bounds on predictions
• ON ESTIMATING THRESHOLDS IN AUTOREGRESSIVE MODELS
• SNOC: A Monte-Carlo simulation package for high-zsupernova observations
• An optimal hydrodynamic model for the normal type IIP supernova 1999em
• Using models to improve optimizers for variational quantum algorithms
• Optimum simple ramp-tests for the Weibull distribution and type-I censoring
• Empirical Characteristic Function Estimation and Its Applications
• Quantum state tomography as a numerical optimization problem
• Anomaly detection from mass unspecific jet tagging
• Adiabatic and local approximations for the Kohn-Sham potential in time-dependent Hubbard chains
• 3D print of polymer bonded rare-earth magnets, and 3D magnetic field scanning with an end-user 3D printer
• Adaptive Circuit Learning for Quantum Metrology
• Automatic observation rendering (AMORE)
• Quantum variational optimization: The role of entanglement and problem hardness
• Optimization of a Controlled-Z Gate with Data-Driven Gradient-Ascent Pulse Engineering in a Superconducting-Qubit System
• Halo-independence with quantified maximum entropy at DAMA/LIBRA
• Ultrafast critical ground state preparation via bang–bang protocols
• Estimating equations for causal survival analysis with pooled logistic regression
• The LSST-DESC 3x2pt Tomography Optimization Challenge
• Retinal Imaging and Image Analysis
• Greedy Restart Schedules: A Baseline for Dynamic Algorithm Selection on Numerical Black-box Optimization Problems
• Imaging the water snow-line during a protostellar outburst
• Stochastic evolution of rationality
• PyCogent: a toolkit for making sense from sequence. [europepmc]
• Evaluating the robustness of phylogenetic methods to among-site variability in substitution processes. [europepmc]
• Membrane bending is critical for the stability of voltage sensor segments in the membrane. [europepmc]
• Continuous roadmapping in liver TACE procedures using 2D-3D catheter-based registration. [europepmc]
• Recalling visual serial order for verbal sequences. [europepmc]
• A Computational Cognitive Biomarker for Early-Stage Huntington's Disease. [europepmc]
• Bypassing the Kohn-Sham equations with machine learning. [europepmc]
• Directed Self-Assembly of Polarizable Ellipsoids in an External Electric Field. [europepmc]
• Discovery of novel choline acetyltransferase inhibitors using structure-based virtual screening. [europepmc]
• Robotic drill guide positioning using known-component 3D-2D image registration. [europepmc]
• Target-Controlled Continuous Infusion for Antibiotic Dosing: Proof-of-Principle in an In-silico Vancomycin Trial in Intensive Care Unit Patients. [europepmc]
• Structure-Activity Relationships Based on 3D-QSAR CoMFA/CoMSIA and Design of Aryloxypropanol-Amine Agonists with Selectivity for the Human β3-Adrenergic Receptor and Anti-Obesity and Anti-Diabetic Profiles. [europepmc]
• American Society of Biomechanics Clinical Biomechanics Award 2017: Non-anatomic graft geometry is linked with asymmetric tibiofemoral kinematics and cartilage contact following anterior cruciate ligament reconstruction. [europepmc]
• Spatial frequency domain imaging using an analytical model for separation of surface and volume scattering. [europepmc]
• Spatial sampling in human visual cortex is modulated by both spatial and feature-based attention. [europepmc]
• Estimating axon conduction velocity in vivo from microstructural MRI. [europepmc]
• Chunking and redintegration in verbal short-term memory. [europepmc]
• Antigenicity prediction and vaccine recommendation of human influenza virus A (H3N2) using convolutional neural networks. [europepmc]
• Predicting evolutionary change at the DNA level in a natural Mimulus population. [europepmc]
• Overly Strong Priors for Socially Meaningful Visual Signals Are Linked to Psychosis Proneness in Healthy Individuals. [europepmc]
• Nucleation mechanisms and speciation of metal oxide clusters. [europepmc]
• Spectral signature and behavioral consequence of spontaneous shifts of pupil-linked arousal in human. [europepmc]
• Fisher Scoring for crossed factor linear mixed models. [europepmc]
• Reverse engineering of metacognition. [europepmc]
• Increasing the Accuracy and Robustness of the CHARMM General Force Field with an Expanded Training Set. [europepmc]

Related

• Function minimization by conjugate gradients
• A seven-point algorithm for piecewise smooth univariate minimization
• Been Down So Long it Looks Like Up to Me: A Unified Derivation of Conjugate Gradient and Variable Metric Minimization
• A Simplex Method for Function Minimization
• A Simple and Efficient Method for Solving Multi-Objective Programming Problems and Multi-Objective Optimal Controls