Distributed stochastic gradient tracking methods

2020/03/17 by Shi Pu, Angelia Nedić · 34 citations
Computer Science · Engineering · #Distributed Control Multi-Agent Systems #Stochastic Gradient Optimization Techniques #Sparse and Compressive Sensing Techniques

paper · doi:10.1007/s10107-020-01487-0

Citations

• A Stochastic Approximation Method
• Stochastic Estimation of the Maximum of a Regression Function
• Constrained Consensus and Optimization in Multi-Agent Networks
• Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling

Cited by

• On the Linear Speedup of the Push-Pull Method for Decentralized Optimization over Digraphs
• Distributed Nash Equilibrium Seeking in N-Cluster Games with Fully Uncoordinated Constant Step-Sizes
• Variance Reduced EXTRA and DIGing and Their Optimal Acceleration for Strongly Convex Decentralized Optimization
• Distributed Online Economic Dispatch with Time-Varying Coupled Inequality Constraints
• An Accelerated Primal Dual Algorithm with Backtracking for Decentralized Constrained Optimization
• Distributed Policy Gradient with Variance Reduction in Multi-Agent Reinforcement Learning
• Beyond Scaffold: A Unified Spatio-Temporal Gradient Tracking Method
• Accelerating Distributed Online Meta-Learning via Multi-Agent Collaboration under Limited Communication
• Row-Stochastic Matrices Can Provably Outperform Doubly Stochastic Matrices in Decentralized Learning
• On Linear Convergence of Distributed Stochastic Bilevel Optimization over Undirected Networks via Gradient Aggregation
• A Differential Private Method for Distributed Optimization in Directed Networks via State Decomposition
• HBNET-GIANT: A communication-efficient accelerated Newton-type fully distributed optimization algorithm
• Event-triggered Dual Gradient Tracking for Distributed Resource Allocation
• Faster Non-Convex Federated Learning via Global and Local Momentum
• Provable local acceleration in HbNet-GIANT: an accelerated Newton-type distributed optimization algorithm
• Guaranteeing Both Consensus and Optimality in Decentralized Nonconvex Optimization with Multiple Local Updates
• Provably Accelerated Decentralized Gradient Method Over Unbalanced Directed Graphs
• Distributed Stochastic Momentum Tracking with Local Updates: Achieving Optimal Communication and Iteration Complexities
• Distributed Stochastic Proximal Algorithm on Riemannian Submanifolds for Weakly-convex Functions
• Convergence Analysis of Noisy Distributed Gradient Descent for Non-convex Optimization -- Saddle Point Escape
• Improved High-probability Convergence Guarantees of Decentralized SGD
• DeMuon: A Decentralized Muon for Matrix Optimization over Graphs
• Distributed Online Convex Optimization with Improved Dynamic Regret
• Removing Data Heterogeneity Influence Enhances Network Topology Dependence of Decentralized SGD
• Distributed Stochastic Optimization under Heavy-Tailed Noise: A Federated Mirror Descent Approach with High Probability Convergence
• A Weighted Gradient Tracking Privacy-Preserving Method for Distributed Optimization
• Pareto-optimal Trade-offs Between Communication and Computation with Flexible Gradient Tracking
• Over-the-Air Decentralized Federated Learning
• Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participation
• Distributed stochastic gradient tracking methods with momentum acceleration for non-convex optimization
• An Optimistic Gradient Tracking Method for Distributed Minimax Optimization
• CoCoL: A Communication Efficient Decentralized Collaborative Method for Multi-Robot Systems
• A Hybrid Stochastic Gradient Tracking Method for Distributed Online Optimization Over Time-Varying Directed Networks
• Decentralized Composite Optimization with Compression
• Improving the Transient Times for Distributed Stochastic Gradient Methods