Tram-FL: Routing-based Model Training for Decentralized Federated Learning

2023/08/09 by Maejima, Kota, Nishio, Takayuki, Yamazaki, Asato +1 · 2 citations
#68T20 (Primary) 68M14 (Secondary) #Distributed #FOS: Computer and information sciences #I.2.10 #I.2.11 #I.2.7 #I.2.8 #Machine Learning (cs.LG) #Networking and Internet Architecture (cs.NI) #Parallel #and Cluster Computing (cs.DC)

paper · doi:10.48550/arxiv.2308.04762

Abstract

In decentralized federated learning (DFL), substantial traffic from frequent inter-node communication and non-independent and identically distributed (non-IID) data challenges high-accuracy model acquisition. We propose Tram-FL, a novel DFL method, which progressively refines a global model by transferring it sequentially amongst nodes, rather than by exchanging and aggregating local models. We also introduce a dynamic model routing algorithm for optimal route selection, aimed at enhancing model precision with minimal forwarding. Our experiments using MNIST, CIFAR-10, and IMDb datasets demonstrate that Tram-FL with the proposed routing delivers high model accuracy under non-IID conditions, outperforming baselines while reducing communication costs.

Cited by

• Load-Aware Training Scheduling for Model Circulation-based Decentralized Federated Learning
• Graph-based Gossiping for Communication Efficiency in Decentralized Federated Learning

Related

• FedRewind: Rewinding Continual Model Exchange for Decentralized Federated Learning
• Route-and-Aggregate Decentralized Federated Learning Under Communication Errors
• Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environments
• The Effect of Training Parameters and Mechanisms on Decentralized Federated Learning based on MNIST Dataset
• Load-Aware Training Scheduling for Model Circulation-based Decentralized Federated Learning