2021/04/11 by Nicolas Lepage-Saucier, Lepage-Saucier, Nicolas
Computer Science · Mathematics · #65K05 #Advanced Optimization Algorithms Research #F.2.1 #FOS: Mathematics #Matrix Theory and Algorithms #Numerical Analysis (math.NA) #Numerical methods for differential equations #Optimization and Control (math.OC)
paper · pdf · doi:10.48550/arxiv.2104.04974
openalex publication_date 2021/04/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
This article introduces new acceleration methods for fixed-point iterations. Extrapolations are computed using two or three mappings alternately and a new type of step length is proposed with good properties for nonlinear applications. The methods require no problem-specific adaptation and are especially efficient in high-dimensional contexts. Their computation uses few objective function evaluations, no matrix inversion and little extra memory. A convergence analysis is followed by eight applications including gradient descent acceleration for constrained and unconstrained optimization. Performances are on par with or better than competitive alternatives. The algorithm is available as the Julia package SpeedMapping.jl.