A Concise Lyapunov Analysis of Nesterov's Accelerated Gradient Method

2025/02/24 by Liu, Jun
#FOS: Computer and information sciences #FOS: Electrical engineering #FOS: Mathematics #Machine Learning (cs.LG) #Optimization and Control (math.OC) #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2502.17373

Abstract

Convergence analysis of Nesterov's accelerated gradient method has attracted significant attention over the past decades. While extensive work has explored its theoretical properties and elucidated the intuition behind its acceleration, a simple and direct proof of its convergence rates is still lacking. We provide a concise Lyapunov analysis of the convergence rates of Nesterov's accelerated gradient method for both general convex and strongly convex functions.

Related

• The Global R-linear Convergence of Nesterov's Accelerated Gradient Method with Unknown Strongly Convex Parameter
• Revisiting the acceleration phenomenon via high-resolution differential equations
• A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights
• Optimal convergence rates for Nesterov acceleration
• Gradient Norm Minimization of Nesterov Acceleration: o(1/k³)