Expanding the Reach of Federated Learning by Reducing Client Resource Requirements

2018/12/18 by Sebastian Caldas, Caldas, Sebastian, Jakub Konečný +5 · 14 citations
Computer Science · Engineering · #Advanced Graph Neural Networks #Distributed #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Parallel #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques #Wireless Communication Security Techniques #and Cluster Computing (cs.DC)

paper · pdf · doi:10.48550/arxiv.1812.07210

openalex publication_date 2018/12/18 · openalex created_date 2022/08/01 · openalex updated_date 2026/07/28

Abstract

Communication on heterogeneous edge networks is a fundamental bottleneck in Federated Learning (FL), restricting both model capacity and user participation. To address this issue, we introduce two novel strategies to reduce communication costs: (1) the use of lossy compression on the global model sent server-to-client; and (2) Federated Dropout, which allows users to efficiently train locally on smaller subsets of the global model and also provides a reduction in both client-to-server communication and local computation. We empirically show that these strategies, combined with existing compression approaches for client-to-server communication, collectively provide up to a 14× reduction in server-to-client communication, a 1.7× reduction in local computation, and a 28× reduction in upload communication, all without degrading the quality of the final model. We thus comprehensively reduce FL's impact on client device resources, allowing higher capacity models to be trained, and a more diverse set of users to be reached.

Cited by

• Energy and Memory-Efficient Federated Learning With Ordered Layer Freezing
• First Provable Guarantees for Practical Private FL: Beyond Restrictive Assumptions
• DFedReweighting: A Unified Framework for Objective-Oriented Reweighting in Decentralized Federated Learning
• CycleSL: Server-Client Cyclical Update Driven Scalable Split Learning
• Coherence-Aware Distributed Learning under Heterogeneous Downlink Impairments
• Federated Learning in Mobile Edge Networks: A Comprehensive Survey
• CLIP: Client-Side Invariant Pruning for Mitigating Stragglers in Secure Federated Learning
• A survey on federated learning
• FedEL: Federated Elastic Learning for Heterogeneous Devices
• Recent advances on federated learning: A systematic survey
• FedAPTA: Federated Multi-task Learning for Heterogeneous Devices with Adaptive Layer-wise Pruning and Task-aware Aggregation
• Boosting Generalization Performance in Model-Heterogeneous Federated Learning Using Variational Transposed Convolution
• Hypernetworks for Model-Heterogeneous Personalized Federated Learning
• Model Parallelism With Subnetwork Data Parallelism

Related

• Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learning
• Federated Learning: Strategies for Improving Communication Efficiency
• FedConv: A Learning-on-Model Paradigm for Heterogeneous Federated Clients
• FedSZ: Leveraging Error-Bounded Lossy Compression for Federated Learning Communications
• FedBWO: Enhancing Communication Efficiency in Federated Learning