2025/01/17 by M.W. Fu, Bin Shi, Fu, Mingwei +1
Computer Science · Engineering · #FOS: Computer and information sciences #FOS: Mathematics #Inertial Sensor and Navigation #Machine Learning (stat.ML) #Numerical Analysis (math.NA) #Optimization and Control (math.OC) #Spacecraft Dynamics and Control #Target Tracking and Data Fusion in Sensor Networks
paper · pdf · doi:10.48550/arxiv.2501.10051
openalex publication_date 2025/01/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Nesterov's accelerated gradient method (NAG) marks a pivotal advancement in gradient-based optimization, achieving faster convergence compared to the vanilla gradient descent method for convex functions. However, its algorithmic complexity when applied to strongly convex functions remains unknown, as noted in the comprehensive review by Chambolle and Pock [2016]. This issue, aside from the critical step size, was addressed by Li et al. [2024b], with the monotonic case further explored by Fu and Shi [2024]. In this paper, we introduce a family of controllable momentum coefficients for forward-backward accelerated methods, focusing on the critical step size s=1/L. Unlike traditional linear forms, the proposed momentum coefficients follow an α-th power structure, where the parameter r is adaptively tuned to α. Using a Lyapunov function specifically designed for α, we establish a controllable O(1/k2α ) convergence rate for the NAG-α method, provided that r > 2α. At the critical step size, NAG-α achieves an inverse polynomial convergence rate of arbitrary degree by adjusting r according to α> 0. We further simplify the Lyapunov function by expressing it in terms of the iterative sequences xk and yk, eliminating the need for phase-space representations. This simplification enables us to extend the controllable O (1/k2α ) rate to the monotonic variant, M-NAG-α, thereby enhancing optimization efficiency. Finally, by leveraging the fundamental inequality for composite functions, we extended the controllable O(1/k2α ) rate to proximal algorithms, including the fast iterative shrinkage-thresholding algorithm (FISTA-α) and its monotonic counterpart (M-FISTA-α).