Adaptive fast gradient method in stochastic optimization tasks

2017/11/30 by Alexander Tyurin, Tyurin, Alexander
Computer Science · Engineering · #FOS: Mathematics #Neural Networks and Applications #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.1712.00062

openalex publication_date 2017/11/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we describe a stochastic adaptive fast gradient descent method based on the mirror variant of similar triangles method. To our knowledge, this is the first attempt to use adaptivity in stochastic method. Additionally, a main result was proved in terms of probabilities of large deviations.

Related

• Adaptive Gradient Descent for Convex and Non-Convex Stochastic Optimization
• Adaptive Learning Rates for Faster Stochastic Gradient Methods
• Adaptive Stochastic Mirror Descent for Constrained Optimization
• Stochastic Adaptive Gradient Descent Without Descent
• Non asymptotic analysis of Adaptive stochastic gradient algorithms and applications