Decentralized Learning over Wireless Networks: The Effect of Broadcast with Random Access

2023/05/12 by Chen, Zheng, Dahl, Martin, Larsson, Erik G. · 1 citation
#FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Networking and Internet Architecture (cs.NI) #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2305.07368

Abstract

In this work, we focus on the communication aspect of decentralized learning, which involves multiple agents training a shared machine learning model using decentralized stochastic gradient descent (D-SGD) over distributed data. In particular, we investigate the impact of broadcast transmission and probabilistic random access policy on the convergence performance of D-SGD, considering the broadcast nature of wireless channels and the link dynamics in the communication topology. Our results demonstrate that optimizing the access probability to maximize the expected number of successful links is a highly effective strategy for accelerating the system convergence.

Cited by

• Ubiquitous Intelligence Via Wireless Network-Driven LLMs Evolution

Related

• Decentralized Learning over Wireless Networks with Broadcast-Based Subgraph Sampling
• Network-Density-Controlled Decentralized Parallel Stochastic Gradient Descent in Wireless Systems
• Faster Convergence with Less Communication: Broadcast-Based Subgraph Sampling for Decentralized Learning over Wireless Networks
• Decentralized Federated Learning via SGD over Wireless D2D Networks
• Asynchronous Decentralized Learning over Unreliable Wireless Networks