Distributed Stochastic Compositional Optimization Problems over Directed Networks

2022/03/21 by Shengchao Zhao, Yongchao Liu, Zhao, Shengchao +1
Computer Science · #Distributed Control Multi-Agent Systems #FOS: Mathematics #Machine Learning and ELM #Optimization and Control (math.OC) #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2203.11074

openalex publication_date 2022/03/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We study the distributed stochastic compositional optimization problems over directed communication networks in which agents privately own a stochastic compositional objective function and collaborate to minimize the sum of all objective functions. We propose a distributed stochastic compositional gradient descent method, where the gradient tracking and the stochastic correction techniques are employed to adapt to the networks' directed structure and increase the accuracy of inner function estimation. When the objective function is smooth, the proposed method achieves the convergence rate O(k^-1/2) and sample complexity O((1)/(ε²)) for finding the (ε)-stationary point. When the objective function is strongly convex, the convergence rate is improved to O(k^-1). Moreover, the asymptotic normality of Polyak-Ruppert averaged iterates of the proposed method is also presented. We demonstrate the empirical performance of the proposed method on model-agnostic meta-learning problem and logistic regression problem.

Related

• Numerical Methods for Distributed Stochastic Compositional Optimization Problems with Aggregative Structure
• Achieving Linear Speedup in Decentralized Stochastic Compositional Minimax Optimization
• Stochastic Subgradient Algorithms for Strongly Convex Optimization over Distributed Networks
• Distributed Stochastic Optimization with Gradient Tracking over Time-Varying Directed Networks
• Nested Distributed Gradient Methods with Stochastic Computation Errors