Convergence in quadratic mean of averaged stochastic gradient algorithms\n without strong convexity nor bounded gradient

2021/07/26 by Antoine Godichon‐Baggioni, Godichon-Baggioni, Antoine
Computer Science · Decision Sciences · Engineering · #Advanced Bandit Algorithms Research #FOS: Mathematics #Sparse and Compressive Sensing Techniques #Statistics Theory (math.ST) #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2107.12058

openalex publication_date 2021/07/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Online averaged stochastic gradient algorithms are more and more studied\nsince (i) they can deal quickly with large sample taking values in high\ndimensional spaces, (ii) they enable to treat data sequentially, (iii) they are\nknown to be asymptotically efficient. In this paper, we focus on giving\nexplicit bounds of the quadratic mean error of the estimates, and this, with\nvery weak assumptions, i.e without supposing that the function we would like to\nminimize is strongly convex or admits a bounded gradient.\n

Citations

• A Stochastic Approximation Method
• Online estimation of the asymptotic variance for averaged stochastic\n gradient algorithms
• Efficient and fast estimation of the geometric median in Hilbert spaces with an averaged stochastic gradient algorithm

Related

• Lp and almost sure rates of convergence of averaged stochastic gradient algorithms: locally strongly convex objective
• Online estimation of the asymptotic variance for averaged stochastic\n gradient algorithms
• Simple and optimal high-probability bounds for strongly-convex\n stochastic gradient descent
• Optimized convergence of stochastic gradient descent by weighted averaging
• On the rates of convergence of Parallelized Averaged Stochastic Gradient\n Algorithms