Asynchronous decentralized accelerated stochastic gradient descent

2018/09/24 by Guanghui Lan, Yi Zhou, Lan, Guanghui +1 · 2 citations
Computer Science · Engineering · #Distributed Control Multi-Agent Systems #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.1809.09258

openalex publication_date 2018/09/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this work, we introduce an asynchronous decentralized accelerated stochastic gradient descent type of method for decentralized stochastic optimization, considering communication and synchronization are the major bottlenecks. We establish O(1/ε) (resp., O(1/√ε)) communication complexity and O(1/ε²) (resp., O(1/ε)) sampling complexity for solving general convex (resp., strongly convex) problems.

Citations

• On the O(1/k) Convergence of Asynchronous Distributed Alternating Direction Method of Multipliers
• A Coordinate Descent Primal-Dual Algorithm and Application to\n Distributed Asynchronous Optimization

Cited by

• Accelerated Sparsified SGD with Error Feedback
• Robust and Communication-Efficient Collaborative Learning

Related

• On the Optimal Time Complexities in Decentralized Stochastic Asynchronous Optimization
• Asynchronous stochastic convex optimization
• Asynchronous Decentralized SGD under Non-Convexity: A Block-Coordinate Descent Framework
• Decentralized and Parallel Primal and Dual Accelerated Methods for Stochastic Convex Programming Problems
• An Accelerated Method For Decentralized Distributed Stochastic\n Optimization Over Time-Varying Graphs