Multiplier and gradient methods

1969/11/01 by Magnus R. Hestenes · 2,472 citations
Computer Science · Mathematics · #Advanced Optimization Algorithms Research #Algorithm #Applied mathematics #Computer science #Conjugate gradient method #Conjugate residual method #Constant (computer programming) #Gradient descent #Gradient method #Iterative Methods for Nonlinear Equations #Lagrange multiplier #Mathematical analysis #Mathematical optimization #Mathematics #Matrix Theory and Algorithms #Maxima and minima #Multiplier (economics) #Theory of computation

paper · doi:10.1007/bf00927673

published in Journal of Optimization Theory and Applications 4(5), 303-320 (Springer Science+Business Media)

openalex publication_date 1969/11/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/04

Cited by

• Multi-block Nonconvex Nonsmooth Proximal ADMM: Convergence and Rates Under Kurdyka–Łojasiewicz Property
• On the Global Convergence of a General Class of Augmented Lagrangian Methods
• The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent
• Modeling of the multi-scale related thermo-mechanical coupling behaviors in a helical cruciform fuel assembly
• Lagrange Multipliers and Optimality
• Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods
• Differentially Private Neural Network Training Under the Hidden State Assumption
• Limiting Stationarity of Regularized Gap-Function Reformulations for Bilevel Optimization with Unbounded Multipliers
• An Efficient Augmented Lagrangian Framework for Dynamic Optimal Transport on Surfaces Based on Second-Order Cone Programming Reformulation
• Hybrid Augmented Lagrangian Method for General Constrained Optimization via Evolutionary Algorithms
• Global Convergence of Inexact Augmented Lagrangian Method for Zero-One Composite Optimization
• Reducing Electricity Demand Charge for Data Centers with Partial Execution
• Multiscale Higher Order TV Operators for L1 Regularization
• Approximating Optimal Transport via Low-rank and Sparse Factorization
• Schwarzite and schwarzynes based load-bear resistant radial cellular griding-based 3D printed structures
• On the Equivalence of Inexact Proximal ALM and ADMM for a Class of Convex Composite Programming
• The Golden Ratio Proximal ADMM with Norm Independent Step-Sizes for Separable Convex Optimization
• Pontryagin-Bellman Differential Dynamic Programming for Low-Thrust Trajectory Optimization with Path Constraints
• Implementation of the ADMM to Parabolic Optimal Control Problems with Control Constraints and Beyond
• Direct inversion of data-space Hessian for efficient time-domain extended-source waveform inversion using the multiplier method
• A Low-rank Augmented Lagrangian Method for Polyhedral-SDP and Moment-SOS Relaxations of Polynomial Optimization
• Deep FlexQP: Accelerated Nonlinear Programming via Deep Unfolding
• Alternating Direction Algorithms for ℓ₁-Problems in Compressive Sensing
• Duality methods for solving variational inequalities
• Optimization of continuous-flow over traffic networks with fundamental diagram constraints
• Augmented Lagrangian methods for fully convex composite optimization
• Complexity of Proximal augmented Lagrangian for nonconvex optimization with nonlinear equality constraints
• Proximal gradient descent on the smoothed duality gap to solve saddle point problems
• Sequential Quadratic Optimization for Nonlinear Equality Constrained Stochastic Optimization
• Optimization of continuous-flow over traffic networks with fundamental diagram constraints
• Alternating Direction Method of Multipliers for Linear Inverse Problems
• A two-level distributed algorithm for nonconvex constrained optimization
• Indefinite linearized augmented Lagrangian method for convex programming with linear inequality constraints
• Estimation for bivariate quantile varying coefficient model
• Iteration complexity of first-order augmented Lagrangian methods for convex conic programming
• Subspace based low rank and joint sparse matrix recovery
• Balanced Augmented Lagrangian Method for Convex Programming
• On Certifying Non-uniform Bound against Adversarial Attacks
• Solving Optimization Problems over the Stiefel Manifold by Smooth Exact Penalty Function
• Zeroth Order Nonconvex Multi-Agent Optimization over Networks
• KKT Conditions, First-Order and Second-Order Optimization, and Distributed Optimization: Tutorial and Survey
• Fast Image Recovery Using Variable Splitting and Constrained Optimization
• Managing Randomization in the Multi-Block Alternating Direction Method of Multipliers for Quadratic Optimization
• Convergence of a Relative-type Inexact Proximal ALM for Convex Nonlinear Programming
• On the Convergence of Primal-Dual Hybrid Gradient Algorithm
• A projection-free dynamics for nonsmooth composite optimization
• A PyTorch Framework for Scalable Non-Crossing Quantile Regression
• Rayleigh Quotient Iteration, cubic convergence, and second covariant derivative
• Nonmonotone subgradient methods based on a local descent lemma
• Communication-Efficient Algorithms For Distributed Optimization
• A Novel Preconditioning Framework for Solving Nonlinear PDEs based on Fenchel-Rockafellar Duality and Transformed Primal-Dual Techniques
• An Augmented Lagrangian Method-Based Framework in the Adjoint Space for Sparse Reconstruction of Acoustic Sources
• Semi-infinite Nonconvex Constrained Min-Max Optimization
• The Rate of Convergence of the Augmented Lagrangian Method for a Nonlinear Semidefinite Nuclear Norm Composite Optimization Problem
• PENEX: AdaBoost-Inspired Neural Network Regularization
• Calibrating Generative Models
• On the Linear and Asymptotically Superlinear Convergence Rates of the Augmented Lagrangian Method with a Practical Relative Error Criterion
• Scalable Kernel Quantile Regression: A Preconditioned Augmented Lagrangian Method
• MadNCL: A GPU Implementation of Algorithm NCL for Large-Scale, Degenerate Nonlinear Programs
• A Sensitivity-Based Method for Bilevel Optimization Problems: Theoretical Analysis and Computational Performance
• Learning to Solve Optimization Problems Constrained with Partial Differential Equations
• Dual Optimistic Ascent (PI Control) is the Augmented Lagrangian Method in Disguise
• Responsive Safety in Reinforcement Learning by PID Lagrangian Methods
• Constructing Deep Neural Networks with a Priori Knowledge of Wireless Tasks
• A two-level distributed algorithm for nonconvex constrained optimization
• On the Optimal Linear Convergence Rate of a Generalized Proximal Point Algorithm
• Regression-based Network Reconstruction with Nodal and Dyadic Covariates and Random Effects
• Subspace Clustering of Subspaces: Unifying Canonical Correlation Analysis and Subspace Clustering
• A Continuous Feedback Optimal Control based on Second-Variations for Problems with Control Constraints
• An Optimal Control Interpretation of Augmented Distributed Optimization Algorithms
• Hybrid Learning and Optimization methods for solving Capacitated Vehicle Routing Problem
• Joint Long-Term Admission Control and Beamforming in Green Downlink Networks: Offline and Online Approaches
• A Class of Algorithms for General Instrumental Variable Models
• Q-Linear Convergence of the Proximal Augmented Lagrangian Method for Non-Convex Conic Programming
• PAL-Hom method for QP and an application to LP
• On the tensor nuclear norm and the total variation regularization for image and video completion
• Fast Proximal Linearized Alternating Direction Method of Multiplier with Parallel Splitting
• A proximal augmented Lagrangian method for nonconvex optimization with equality and inequality constraints
• User Manual for Model-based Imaging Inverse Problem
• Escape over a saddle by coloured noise: theory and numerics
• Low-Rank Regularized Convex-Non-Convex Problems for Image Segmentation or Completion
• Reflection Resource Management for Intelligent Reflecting Surface Aided Wireless Networks
• Conditionally adaptive augmented Lagrangian method for physics-informed learning of forward and inverse problems
• A dual-primal balanced augmented Lagrangian method for linearly constrained convex programming
• DOPE: Distributed Optimization for Pairwise Energies
• A simple and efficient algorithm for fused lasso signal approximator with convex loss function
• A Splitting Augmented Lagrangian Method for Low Multilinear-Rank Tensor Recovery
• An Alternating Direction Method Approach to Cloud Traffic Management
• Mean Field Type Control with Congestion (II): An Augmented Lagrangian Method
• A family of multi-parameterized proximal point algorithms
• A Two-level ADMM Algorithm for AC OPF with Global Convergence Guarantees
• Anomaly Detection via Graphical Lasso
• Monotone Operators and the Proximal Point Algorithm
• Extended Factorization Machine Annealing for Rapid Discovery of Transparent Conducting Materials
• The Augmented Mixing Method: Computing High-Accuracy Primal-Dual Solutions to Large-Scale SDPs via Column Updates
• Convergence of dual ascent in non-convex/non-differentiable optimization
• Physics-Informed Regression: Parameter Estimation in Parameter-Linear Nonlinear Dynamic Models
• Optimal Nonergodic Sublinear Convergence Rate of Proximal Point Algorithm for Maximal Monotone Inclusion Problems
• Hyperparameter Analysis for Derivative Compressive Sampling
• MADMM: a generic algorithm for non-smooth optimization on manifolds
• Efficient Latent Variable Graphical Model Selection via Split Bregman Method
• A corrected semi-proximal ADMM for multi-block convex optimization and its application to DNN-SDPs
• An Integrated Inverse Space Sparse Representation Framework for Tumor Classification
• An augmented Lagrangian method for strongly regular minimizers in a class of convex composite optimization problems
• Cutting Slack: Quantum Optimization with Slack-Free Methods for Combinatorial Benchmarks
• High-Fidelity Machine Learning Approximations of Large-Scale Optimal Power Flow
• A Bi-Orthogonal Structure-Preserving eigensolver for large-scale linear response eigenvalue problem
• Estimation of low-rank tensors via convex optimization
• Decomposing Linearly Constrained Nonconvex Problems by a Proximal Primal Dual Approach: Algorithms, Convergence, and Applications
• Joint Variable Selection for Data Envelopement Analysis via Group Sparsity
• The Capacity Constraint Physarum Solver
• Inexact indefinite proximal ADMMs for 2-block separable convex programs and applications to 4-block DNNSDPs
• Parallel multilevel methods for solving the Darcy--Forchheimer model based on a nearly semicoercive formulation
• Image Restoration using Total Variation with Overlapping Group Sparsity
• Algorithms for Overcoming the Curse of Dimensionality for Certain Hamilton-Jacobi Equations Arising in Control Theory and Elsewhere
• Augmented Lagrangian methods for infeasible convex optimization problems and diverging proximal-point algorithms
• Higher-degree eigenvalue complementarity problems for tensors
• On the R-superlinear convergence of the KKT residues generated by the augmented Lagrangian method for convex composite conic programming
• A General System for Heuristic Solution of Convex Problems over Nonconvex Sets
• ConStellaration: A dataset of QI-like stellarator plasma boundaries and optimization benchmarks
• Convergent Proximal Multiblock ADMM for Nonconvex Dynamics-Constrained Optimization
• On the Convergence Analysis of Asynchronous Distributed Quadratic\n Programming via Dual Decomposition
• A distributed primal-dual interior-point method for loosely coupled problems using ADMM
• Coupled Learning for Facial Deblur
• Image Restoration and Reconstruction using Variable Splitting and Class-adapted Image Priors
• On the pure state v-representability of density matrix embedding theory
• Toeplitz Inverse Covariance-Based Clustering of Multivariate Time Series Data
• Inexact JKO and proximal-gradient algorithms in the Wasserstein space
• Alternating Directions Dual Decomposition
• Algorithms for Difference-of-Convex (DC) Programs Based on Difference-of-Moreau-Envelopes Smoothing
• An Exact Penalty Approach for Equality Constrained Optimization over a Convex Set
• An accelerated semi-proximal ADMM with applications to multi-block sparse optimization problems
• Local Neighborhood Fusion in Locally Constant Gaussian Graphical Models
• Position: Adopt Constraints Over Fixed Penalties in Deep Learning
• MorteX method for contact along real and embedded surfaces: coupling X-FEM with the Mortar method
• Random multi-block ADMM: an ALM based view for the QP case
• On a Randomized Multi-Block ADMM for Solving Selected Machine Learning Problems
• Compressive Conjugate Directions: Linear Theory
• Dual Descent ALM and ADMM
• On the non-ergodic convergence rate of an inexact augmented Lagrangian framework for composite convex programming
• A Generalized Alternating Direction Method of Multipliers with Semi-Proximal Terms for Convex Composite Conic Programming
• Alternating direction method of multipliers with variable step sizes
• Weighted Lagrange Multiplier Method for Robust Source-Independent Waveform Inversion
• New Understandings and Computation on Augmented Lagrangian Methods for Low-Rank Semidefinite Programming
• A message-passing algorithm for multi-agent trajectory planning
• Stochastic Primal-Dual Coordinate Method for Nonlinear Convex Cone Programs
• Accelerated Inexact First-Order Methods for Solving Nonconvex Composite Optimization Problems
• Separable Approximations and Decomposition Methods for the Augmented\n Lagrangian
• Lagrange Oscillatory Neural Networks for Constraint Satisfaction and Optimization
• An efficient second-order cone programming approach for dynamic optimal transport on staggered grid discretization
• Smoothness of the Augmented Lagrangian Dual in Convex Optimization
• Message Passing for Dynamic Network Energy Management
• NPASA: An algorithm for nonlinear programming -- Motivation and Global Convergence
• Local convergence analysis of augmented Lagrangian method for nonlinear semidefinite programming
• Auxiliary Problem Principle of augmented Lagrangian with Varying Core Functions for Large-Scale Structured Convex Problems
• Iteration complexity analysis of a partial LQP-based alternating direction method of multipliers
• Training Language Models Using Target-Propagation
• An augmented lagrangian treatment of contact problems involving friction
• A mixed formulation for frictional contact problems prone to Newton like solution methods
• Decomposition Methods for Global Solutions of Mixed-Integer Linear Programs
• Optimal Nonergodic Primal-Dual Complexity of Efficient Inexact Parameter-Free Augmented Lagrangian Methods
• Solution of the Skyrme–Hartree–Fock–Bogolyubov equations in the Cartesian deformed harmonic-oscillator basis.
• An Alternating Direction Method for Total Variation Denoising
• Split Bregman Method for Sparse Inverse Covariance Estimation with Matrix Iteration Acceleration
• An alternating direction algorithm for matrix completion with nonnegative factors
• Inertial primal-dual algorithms for structured convex optimization
• Newton-Raphson Consensus for Distributed Convex Optimization
• Fast X-Ray CT Image Reconstruction Using a Linearized Augmented Lagrangian Method With Ordered Subsets
• Preconditioning ideas for the Augmented Lagrangian method
• A New Compressive Video Sensing Framework for Mobile Broadcast
• The first open channel for yield-stress fluids in porous media
• Adaptive cluster approximation for reduced density-matrix functional theory
• FlexPD: A Flexible Framework Of First-Order Primal-Dual Algorithms for Distributed Optimization
• Variational Multiscale Nonparametric Regression: Algorithms and Implementation
• Accelerated Linearized Bregman Method
• Stochastic Dual Coordinate Ascent with Alternating Direction Multiplier Method
• Sparse Wavelet Estimation in Quantile Regression with Multiple Functional Predictors
• A Damped Subspace Splitting Algorithm for Constrained Density Functional Theory
• Stochastic Smoothed Primal-Dual Algorithms for Nonconvex Optimization with Linear Inequality Constraints
• Hierarchical Convex Optimization by the Hybrid Steepest Descent Method with Proximal Splitting Operators—Enhancements of SVM and Lasso
• An Efficient Augmented Lagrangian Method for Statistical X-Ray CT Image Reconstruction. [europepmc]
• Low Dose PET Image Reconstruction with Total Variation Using Alternating Direction Method. [europepmc]
• An accelerated proximal augmented Lagrangian method and its application in compressive sensing. [europepmc]
• Modified hybrid decomposition of the augmented Lagrangian method with larger step size for three-block separable convex programming. [europepmc]
• Multiscale higher-order TV operators for L1 regularization. [europepmc]
• Optimization of Traced Neuron Skeleton Using Lasso-Based Model. [europepmc]
• Density Structure of the Von Kármán Crater in the Northwestern South Pole-Aitken Basin: Initial Subsurface Interpretation of the Chang'E-4 Landing Site Region. [europepmc]
• Method for Distinguishing Humans and Animals in Vital Signs Monitoring Using IR-UWB Radar. [europepmc]
• Single pixel hyperspectral bioluminescence tomography based on compressive sensing. [europepmc]
• 3D Tensor Based Nonlocal Low Rank Approximation in Dynamic PET Reconstruction. [europepmc]
• A Hybrid Fault Diagnosis Approach for Rotating Machinery with the Fusion of Entropy-Based Feature Extraction and SVM Optimized by a Chaos Quantum Sine Cosine Algorithm. [europepmc]
• Mechanical Fault Diagnosis of a DC Motor Utilizing United Variational Mode Decomposition, SampEn, and Random Forest-SPRINT Algorithm Classifiers. [europepmc]
• Simultaneous diffuse optical and bioluminescence tomography to account for signal attenuation to improve source localization. [europepmc]
• Joint Lp-Norm and L 2,1 -Norm Constrained Graph Laplacian PCA for Robust Tumor Sample Clustering and Gene Network Module Discovery. [europepmc]
• Optimization-Based Resource Management Algorithms with Considerations of Client Satisfaction and High Availability in Elastic 5G Network Slices. [europepmc]
• A Total Lp-Norm Optimization for Bearing-Only Source Localization in Impulsive Noise with SαS Distribution. [europepmc]
• Image Denoising Using a Compressive Sensing Approach Based on Regularization Constraints. [europepmc]
• Spontaneous symmetry breaking for extreme vorticity and strain in the three-dimensional Navier-Stokes equations. [europepmc]
• The linearized alternating direction method of multipliers for low-rank and fused LASSO matrix regression model. [europepmc]
• Epileptic Seizure Detection Based on Variational Mode Decomposition and Deep Forest Using EEG Signals. [europepmc]
• QAL-BP: an augmented Lagrangian quantum approach for bin packing. [europepmc]
• Analysis of frequency domain features for the classification of evoked emotions using EEG signals. [europepmc]
• Portability of short term wind power forecasting: investigating model calibration using wind power data from Ireland and UK. [europepmc]
• An sEMG Denoising Method with Improved Threshold Estimation for Rapid Keystroke Tasks [europepmc]