2021/02/11 by Raphaël Berthier, Francis Bach, Francis R. Bach +8 · 1 citation
Computer Science · Engineering · Mathematics · #Distributed #FOS: Computer and information sciences #FOS: Mathematics #Optimization and Control (math.OC) #Parallel #Quantum Information and Cryptography #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques #and Cluster Computing (cs.DC) #cs.DC #math.OC
paper · pdf · doi:10.48550/arxiv.2102.06035
arxiv created 2021/02/11 · openalex publication_date 2021/02/11 · arxiv updated 2021/02/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We introduce the "continuized" Nesterov acceleration, a close variant of Nesterov acceleration whose variables are indexed by a continuous time parameter. The two variables continuously mix following a linear ordinary differential equation and take gradient steps at random times. This continuized variant benefits from the best of the continuous and the discrete frameworks: as a continuous process, one can use differential calculus to analyze convergence and obtain analytical expressions for the parameters; but a discretization of the continuized process can be computed exactly with convergence rates similar to those of Nesterov original acceleration. We show that the discretization has the same structure as Nesterov acceleration, but with random parameters.