Robust Asynchronous Stochastic Gradient-Push: Asymptotically Optimal and Network-Independent Performance for Strongly Convex Functions

2018/11/09 by Artin Spiridonoff, Alex Olshevsky, Spiridonoff, Artin +3 · 28 citations
Computer Science · Engineering · Mathematics · #Applied mathematics #Artificial neural network #Asymptotically optimal algorithm #Asynchronous communication #Bounded function #Combinatorics #Computer science #Convex function #Convex optimization #Discrete mathematics #FOS: Mathematics #Function (biology) #Gradient descent #Lipschitz continuity #Mathematical analysis #Mathematical optimization #Mathematics #Node (physics) #Optimization and Control (math.OC) #Physics #Privacy-Preserving Technologies in Data #Pure mathematics #Regular polygon #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques #Telecommunications #math.OC

paper · pdf · doi:10.48550/arxiv.1811.03982

published in CIPD eBooks 21

openalex publication_date 2018/11/09 · arxiv created 2019/12/29 · arxiv updated 2020/01/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

) at each step.

Cited by

• Historical Information Accelerates Decentralized Optimization: A Proximal Bundle Method
• Machine Learning and CPU (Central Processing Unit) Scheduling Co-Optimization over a Network of Computing Centers
• Delay-Tolerant Augmented-Consensus-based Distributed Directed Optimization
• Momentum-based Accelerated Algorithm for Distributed Optimization under Sector-Bound Nonlinearity
• Stochastic Push-Pull for Decentralized Nonconvex Optimization
• Local SGD With a Communication Overhead Depending Only on the Number of Workers
• A Sharp Estimate on the Transient Time of Distributed Stochastic Gradient Descent
• Fully Asynchronous Distributed Optimization with Linear Convergence in Directed Networks
• Using Non-Lipschitz Signum-Based Functions for Distributed Optimization and Machine Learning: Trade-Off Between Convergence Rate and Optimality Gap
• A Survey of Distributed Optimization Methods for Multi-Robot Systems
• Asynchronous Decentralized SGD under Non-Convexity: A Block-Coordinate Descent Framework
• Advances in Asynchronous Parallel and Distributed Optimization

Related

• Fully Asynchronous Distributed Optimization with Linear Convergence in Directed Networks
• Robust Distributed Accelerated Stochastic Gradient Methods for Multi-Agent Networks
• Push-Pull Gradient Methods for Distributed Optimization in Networks
• Asynchronous Accelerated Proximal Stochastic Gradient for Strongly Convex Distributed Finite Sums
• Asynchronous Stochastic Optimization Robust to Arbitrary Delays