On the rates of convergence of Parallelized Averaged Stochastic Gradient\n Algorithms

2017/10/22 by Antoine Godichon‐Baggioni, Godichon-Baggioni, Antoine, Sofiane Saadane +1
Computer Science · Engineering · Mathematics · #FOS: Mathematics #Markov Chains and Monte Carlo Methods #Sparse and Compressive Sensing Techniques #Statistics Theory (math.ST) #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.1710.07926

openalex publication_date 2017/10/22 · openalex created_date 2022/10/01 · openalex updated_date 2026/07/28

Abstract

The growing interest for high dimensional and functional data analysis led in\nthe last decade to an important research developing a consequent amount of\ntechniques. Parallelized algorithms, which consist in distributing and treat\nthe data into different machines, for example, are a good answer to deal with\nlarge samples taking values in high dimensional spaces. We introduce here a\nparallelized averaged stochastic gradient algorithm, which enables to treat\nefficiently and recursively the data, and so, without taking care if the\ndistribution of the data into the machines is uniform. The rate of convergence\nin quadratic mean as well as the asymptotic normality of the parallelized\nestimates are given, for strongly and locally strongly convex objectives.\n

Related

• Lp and almost sure rates of convergence of averaged stochastic gradient algorithms: locally strongly convex objective
• Convergence in quadratic mean of averaged stochastic gradient algorithms\n without strong convexity nor bounded gradient
• Online estimation of the asymptotic variance for averaged stochastic\n gradient algorithms
• Parallel algorithms and probability of large deviation for stochastic\n optimization problems
• Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization