Dynamic behavior for a gradient algorithm with energy and momentum

2022/03/23 by Hailiang Liu, Liu, Hailiang, Xuping Tian +1
Mathematics · #65K10 (Primary) 90C15 (Secondary) #Classical Analysis and ODEs (math.CA) #FOS: Mathematics #Mathematical Biology Tumor Growth #Optimization and Control (math.OC)

paper · pdf · doi:10.48550/arxiv.2203.12199

openalex publication_date 2022/03/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper investigates a novel gradient algorithm, AGEM, using both energy and momentum, for addressing general non-convex optimization problems. The solution properties of the AGEM algorithm, including aspects such as uniformly boundedness and convergence to critical points, are examined. The dynamic behavior is studied through a comprehensive analysis of a high-resolution ODE system. This ODE system, being nonlinear, is derived by taking the limit of the discrete scheme while preserving the momentum effect through a rescaling of the momentum parameter. The paper emphasizes the global well-posedness of the ODE system and the time-asymptotic convergence of solution trajectories. Furthermore, we establish a linear convergence rate for objective functions that adhere to the Polyak-Łojasiewicz condition.

Related

• A Globally Convergent Gradient Method with Momentum
• Stochastic optimization with momentum: convergence, fluctuations, and\n traps avoidance
• Robustly Stable Accelerated Momentum Methods With A Near-Optimal L2 Gain and H_∞ Performance
• Convergence of First-Order Algorithms with Momentum from the Perspective of an Inexact Gradient Descent Method
• Convergence of a Stochastic Gradient Method with Momentum for Non-Smooth Non-Convex Optimization