A New Perspective of Accelerated Gradient Methods: The Controlled Invariant Manifold Approach

2023/05/18 by Gunjal, Revati, Wagh, Sushama, Nayyer, Syed Shadab +2 · 1 citation
#FOS: Mathematics #Optimization and Control (math.OC)

paper · doi:10.48550/arxiv.2305.10756

Abstract

Gradient Descent (GD) is a ubiquitous algorithm for finding the optimal solution to an optimization problem. For reduced computational complexity, the optimal solution x^* of the optimization problem must be attained in a minimum number of iterations. For this objective, the paper proposes a genesis of an accelerated gradient algorithm through the controlled dynamical system perspective. The objective of optimally reaching the optimal solution x^* where ∇ f(x^*)=0 with a given initial condition x(0) is achieved through control.

Cited by

• On the Dynamics of Acceleration in First order Gradient Methods

Related

• On the Curved Geometry of Accelerated Optimization
• Accelerated First-Order Methods: Differential Equations and Lyapunov\n Functions
• Accelerated Optimization in the PDE Framework: Formulations for the Active Contour Case
• Accelerated Gradient Descent Escapes Saddle Points Faster than Gradient Descent
• Multiscale analysis of accelerated gradient methods