FedHeN: Federated Learning in Heterogeneous Networks

2022/07/07 by Acar, Durmus Alp Emre, Saligrama, Venkatesh
#Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel #and Cluster Computing (cs.DC)

paper · doi:10.48550/arxiv.2207.03031

Abstract

We propose a novel training recipe for federated learning with heterogeneous networks where each device can have different architectures. We introduce training with a side objective to the devices of higher complexities to jointly train different architectures in a federated setting. We empirically show that our approach improves the performance of different architectures and leads to high communication savings compared to the state-of-the-art methods.

Related

• Coordination-free Decentralised Federated Learning on Complex Networks: Overcoming Heterogeneity
• HADFL: Heterogeneity-aware Decentralized Federated Learning Framework
• Ampere: Communication-Efficient and High-Accuracy Split Federated Learning
• Federated Learning based on Pruning and Recovery
• Federated Learning: Challenges, Methods, and Future Directions