Aggregation Delayed Federated Learning

2021/08/17 by Xue Ye, Ye Xue, Xue, Ye +4
Computer Science · #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel #Privacy-Preserving Technologies in Data #and Cluster Computing (cs.DC) #cs.DC #cs.LG

paper · pdf · doi:10.48550/arxiv.2108.07433

arxiv created 2021/08/17 · openalex publication_date 2021/08/17 · arxiv updated 2021/08/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Federated learning is a distributed machine learning paradigm where multiple data owners (clients) collaboratively train one machine learning model while keeping data on their own devices. The heterogeneity of client datasets is one of the most important challenges of federated learning algorithms. Studies have found performance reduction with standard federated algorithms, such as FedAvg, on non-IID data. Many existing works on handling non-IID data adopt the same aggregation framework as FedAvg and focus on improving model updates either on the server side or on clients. In this work, we tackle this challenge in a different view by introducing redistribution rounds that delay the aggregation. We perform experiments on multiple tasks and show that the proposed framework significantly improves the performance on non-IID data.

Related

• An Aggregation-Free Federated Learning for Tackling Data Heterogeneity
• Federated Learning for Non-IID Data via Client Variance Reduction and Adaptive Server Update
• Federated Learning under Heterogeneous and Correlated Client Availability
• Just-in-Time Aggregation for Federated Learning
• FedDRL: Deep Reinforcement Learning-based Adaptive Aggregation for Non-IID Data in Federated Learning