Universal gradient descent

2017/11/01 by Alexander Gasnikov, Gasnikov, Alexander · 1 citation
Computer Science · Engineering · Mathematics · #Advanced Optimization Algorithms Research #FOS: Mathematics #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.1711.00394

openalex publication_date 2017/11/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this book we collect many different and useful facts around gradient descent method. First of all we consider gradient descent with inexact oracle. We build a general model of optimized function that include composite optimization approach, level's methods, proximal methods etc. Then we investigate primal-dual properties of the gradient descent in general model set-up. At the end we generalize method to universal one.

Citations

• A Universal Catalyst for First-Order Optimization
• Smooth minimization of non-smooth functions
• Primal-dual subgradient methods for convex problems
• Computational Optimal Transport: Complexity by Accelerated Gradient Descent Is Better Than by Sinkhorn's Algorithm
• Beyond Convexity: Stochastic Quasi-Convex Optimization

Cited by

• Power-Constrained Policy Gradient Methods for LQR

Related

• Universal Complexity Bounds for Universal Gradient Methods in Nonlinear Optimization
• An overview of gradient descent optimization algorithms
• Universal Intermediate Gradient Method for Convex Problems with Inexact Oracle
• The Physical Systems Behind Optimization Algorithms
• Gradient Methods with Dynamic Inexact Oracles