FedBalancer: Data and Pace Control for Efficient Federated Learning on Heterogeneous Clients

2022/01/05 by Jaemin Shin, Shin, Jaemin, Yuanchun Li +5 · 8 citations
Computer Science · #Age of Information Optimization #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning in Healthcare #Privacy-Preserving Technologies in Data #cs.LG

paper · pdf · doi:10.48550/arxiv.2201.01601

Accepted to the 20th ACM International Conference on Mobile Systems, Applications, and Services (MobiSys 2022)

openalex publication_date 2022/01/05 · arxiv created 2022/05/26 · arxiv updated 2022/05/27 · openalex created_date 2022/06/13 · openalex updated_date 2026/07/28

Abstract

Federated Learning (FL) trains a machine learning model on distributed clients without exposing individual data. Unlike centralized training that is usually based on carefully-organized data, FL deals with on-device data that are often unfiltered and imbalanced. As a result, conventional FL training protocol that treats all data equally leads to a waste of local computational resources and slows down the global learning process. To this end, we propose FedBalancer, a systematic FL framework that actively selects clients' training samples. Our sample selection strategy prioritizes more "informative" data while respecting privacy and computational capabilities of clients. To better utilize the sample selection to speed up global training, we further introduce an adaptive deadline control scheme that predicts the optimal deadline for each round with varying client training data. Compared with existing FL algorithms with deadline configuration methods, our evaluation on five datasets from three different domains shows that FedBalancer improves the time-to-accuracy performance by 1.20~4.48x while improving the model accuracy by 1.1~5.0%. We also show that FedBalancer is readily applicable to other FL approaches by demonstrating that FedBalancer improves the convergence speed and accuracy when operating jointly with three different FL algorithms.

Cited by

• Federated Learning Framework for Scalable AI in Heterogeneous HPC and Cloud Environments
• FedHybrid: Breaking the Memory Wall of Federated Learning via Hybrid Tensor Management
• FLAMMABLE: A Multi-Model Federated Learning Framework with Multi-Model Engagement and Adaptive Batch Sizes
• PQFed: A Privacy-Preserving Quality-Controlled Federated Learning Framework
• PracMHBench: Re-evaluating Model-Heterogeneous Federated Learning Based on Practical Edge Device Constraints
• Toward Storage-Aware Learning with Compressed Data An Empirical Exploratory Study on JPEG
• HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
• ATR-Bench: A Federated Learning Benchmark for Adaptation, Trust, and Reasoning

Related

• FedPrune: Towards Inclusive Federated Learning
• Aergia: Leveraging Heterogeneity in Federated Learning Systems
• Optimization Methods and Software for Federated Learning
• Dynamic Attention-based Communication-Efficient Federated Learning
• Federated Learning under Heterogeneous and Correlated Client Availability