2019/05/17 by Michael Muehlebach, Michael I. Jordan, Muehlebach, Michael +1 · 6 citations
Mathematics · Physics and Astronomy · #FOS: Computer and information sciences #FOS: Electrical engineering #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Mechanical and Optical Resonators #Model Reduction and Neural Networks #Numerical methods for differential equations #Optimization and Control (math.OC) #Systems and Control (eess.SY) #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.1905.07436
openalex publication_date 2019/05/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We present a dynamical system framework for understanding Nesterov's accelerated gradient method. In contrast to earlier work, our derivation does not rely on a vanishing step size argument. We show that Nesterov acceleration arises from discretizing an ordinary differential equation with a semi-implicit Euler integration scheme. We analyze both the underlying differential equation as well as the discretization to obtain insights into the phenomenon of acceleration. The analysis suggests that a curvature-dependent damping term lies at the heart of the phenomenon. We further establish connections between the discretized and the continuous-time dynamics.