Communication-Efficient Learning of Deep Networks from Decentralized Data
2016/02/17 by H. Brendan McMahan, Eider Moore, Daniel Ramage +2 · 2 voices · 92 citations
Computer Science · #cs.LG
paper · pdf
arxiv published 2016/02/17 · arxiv updated 2023/01/26
Abstract
Modern mobile devices have access to a wealth of data suitable for learning models, which in turn can greatly improve the user experience on the device. For example, language models can improve speech recognition and text entry, and image models can automatically select good photos. However, this rich data is often privacy sensitive, large in quantity, or both, which may preclude logging to the data center and training there using conventional approaches. We advocate an alternative that leaves the training data distributed on the mobile devices, and learns a shared model by aggregating locally-computed updates. We term this decentralized approach Federated Learning. We present a practical method for the federated learning of deep networks based on iterative model averaging, and conduct an extensive empirical evaluation, considering five different model architectures and four datasets. These experiments demonstrate the approach is robust to the unbalanced and non-IID data distributions that are a defining characteristic of this setting. Communication costs are the principal constraint, and we show a reduction in required communication rounds by 10-100x as compared to synchronized stochastic gradient descent.
Cited by
- Secure Cluster-Based Hierarchical Federated Learning in Vehicular Networks
- Federated Adapter on Foundation Models: An Out-Of-Distribution Approach
- SemSpaceFL: A Collaborative Hierarchical Federated Learning Framework for Semantic Communication in 6G LEO Satellites
- Preserving Privacy and Utility in LLM-Based Product Recommendations
- A Tight Theory of Error Feedback Algorithms in Distributed Optimization
- Potentials and Pitfalls of Applying Federated Learning in Hardware Assurance
- Federated generative event models for tokenized electronic health records
- Diffusion-Guided Semantic Consistency for Multimodal Heterogeneity
- PFAdapter: Hierarchical LoRA Decomposition for Personalized Federated MLLMs
- MedCL-Bench: Benchmarking stability-efficiency trade-offs and scaling in biomedical continual learning
- The Role of Artificial Intelligence in the SKA Era
- Stealth Signals: Multi-Discriminator GANs for Covert Communications Against Diverse Wardens
- SacFL: Self-Adaptive Federated Continual Learning for Resource-Constrained End Devices
- Communication-Efficient Wireless Federated Fine-Tuning for Large-Scale AI Models
- FedEMA: Federated Exponential Moving Averaging with Negative Entropy Regularizer in Autonomous Driving
- Distributed Human Identity: AI-Enabled Multi-Existence Through Cognitive Replication and Robotic Embodiments
- A bibliometric retrospective of Computers & Security
- Privacy-preserving federated tensor decomposition of single-cell immune data: recovering multicellular programs across institutions
- Distributed Experimental Design: Bayes-optimal Fusion of Local Designs
- Privacy-Preserving Federated Autoencoder for ECG Anomaly Detection on Edge Devices
- Graph Privacy: A Heterogeneous Federated GNN for Trans-Border Financial Data Circulation
- Online Federation For Mixtures of Proprietary Agents with Black-Box Encoders
- FML-bench: A Controlled Study of AI Research Agent Strategies from the Perspective of Search Dynamics
- StreamSplit: Continuous Audio Representation Learning via Uncertainty-Guided Adaptive Splitting
- FedHENet: A Frugal Federated Learning Framework for Heterogeneous Environments
- Sparsification Under Siege: Defending Against Poisoning Attacks in Communication-Efficient Federated Learning
- A Generalized Meta Federated Learning Framework with Theoretical Convergence Guarantees
- Bilateral Differentially Private Vertical Federated Boosted Decision Trees
- Learning Heterogeneous Performance-Fairness Trade-offs in Federated Learning
- FedHERO: A Federated Learning Approach for Node Classification Task on Heterophilic Graphs
- MemArena: An Ego-Centric Benchmark for On-Device Agentic Personal Memory Assistants at Scale
- MetaMoE: Diversity-Aware Proxy Selection for Privacy-Preserving Mixture-of-Experts Unification
- Separating Intelligence from Inference: A Standard for Edge-Native AI Computing
- Relaxed Choices in Bottom-Up Asynchronous Multiparty Session Types
- A Survey on Parameter-Efficient Fine-Tuning for Foundation Models in Federated Learning
- Hubs and Spokes Learning: Efficient and Scalable Collaborative Machine Learning
- FedMVP: Federated Multimodal Visual Prompt Tuning for Vision-Language Models
- Federated learning, ethics, and the double black box problem in medical AI
- Learning and Generalization with Mixture Data
- Clustering-Based Evolutionary Federated Multiobjective Optimization and Learning
- Federated One-Shot Learning with Data Privacy and Objective-Hiding
- Token-Level Prompt Mixture with Parameter-Free Routing for Federated Domain Generalization
- FedCCL: Federated Clustered Continual Learning Framework for Privacy-focused Energy Forecasting
- A Unified Benchmark of Federated Learning with Kolmogorov-Arnold Networks for Medical Imaging
- Soft-Label Caching and Sharpening for Communication-Efficient Federated Distillation
- Who Trains Matters: Federated Learning under Enrollment and Participation Selection Biases
- Scaling Real-Time Traffic Analytics on Edge-Cloud Fabrics for City-Scale Camera Networks
- Robust Federated Personalised Mean Estimation for the Gaussian Mixture Model
- Analysis of Asynchronous Federated Learning: Unraveling the Interactions between Gradient Compression, Delay, and Data Heterogeneity
- FedAvgen: Metadata for Model Aggregation In Communication Systems
- Distributed Reconstruction from Compressive Measurements: Nonconvexity and Heterogeneity
- A False Sense of Privacy: Evaluating Textual Data Sanitization Beyond Surface-level Privacy Leakage
- Federated Out-of-Distribution Generalization: A Causal Augmentation View
- AI Supply Chains: An Emerging Ecosystem of AI Actors, Products, and Services
- Privacy-Preserving Federated Embedding Learning for Localized Retrieval-Augmented Generation
- Harmonizing Generalization and Personalization in Ring-topology Decentralized Federated Learning
- Federation over Text: Insight Sharing for Multi-Agent Reasoning
- Federated Learning with Quantum Enhanced LSTM for Applications in High Energy Physics
- Rashomon Sets and Model Multiplicity in Federated Learning
- UnifyFL: Enabling Decentralized Cross-Silo Federated Learning
- Pseudo-Asynchronous Local SGD: Robust and Efficient Data-Parallel Training
- Privacy-Preserving Personalized Federated Learning for Distributed Photovoltaic Disaggregation under Statistical Heterogeneity
- Federated Client-tailored Adapter for Medical Image Segmentation
- Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima
- Fed-ComBat: A Generalized Federated Framework for Batch Effect Harmonization in Collaborative Studies
- Beyond Peak TOPS/W: A System-Level Perspective on Hybrid Digital, Analogue and Neuromorphic Computing
- FraQ: Efficient Coordinate-Space Recompression for Federated Low-Rank Adaptation
- Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks
- FedCARE: A Multi-Objective Personalised Federated Learning Framework for Smart Healthcare
- FedCritic-MIMO: Communication-Efficient Serverless Federated Critic Learning for Massive-MIMO Resource Control in Open and Disaggregated 6G RANs
- Federated Learning: A Survey on Privacy-Preserving Collaborative Intelligence
- Anti-adversarial Learning: Desensitizing Prompts for Large Language Models
- Generalization Guarantees for Multi-View Representation Learning and Application to Regularization via Gaussian Product Mixture Prior
- Switch-Based Multi-Part Neural Network
- TACO: Tackling Over-correction in Federated Learning with Tailored Adaptive Correction
- Cross-region Model Training with Communication-Computation Overlapping and Delay Compensation
- Decentralized Time Series Classification with ROCKET Features
- GRANITE : a Byzantine-Resilient Dynamic Gossip Learning Framework
- Attention, Anomalies! Handling Attention Layers in Unsupervised Federated Outlier Detection
- Personalized Federated Sparse Adaptation of Time-Series Foundation Models
- On the Effectiveness of Adaptation Strategies for VLM-Based Federated Learning in Remote Sensing
- EvolveNet: Collaborative Harness Evolution for Agent Self-Improvement
- Robust and Personalized Federated Learning for Aircraft-Engine Prognostics under Benign and Adversarial Client Heterogeneity
- FBID: Adaptive Personalized Federated Learning for Robust Out-of-Distribution Attack Detection in IoT Networks
- Toward Federated Large Language Models in Medicine: A Parameter-Efficient Framework for Privacy-Preserving, Multi-Institutional Adaptation
- Federated Learning of Low-Rank One-Shot Image Detection Models in Edge Devices with Scalable Accuracy and Compute Complexity
- Efficient Data Valuation Approximation in Federated Learning: A Sampling-based Approach
- DP2FL: Dual Prompt Personalized Federated Learning in Foundation Models
- Federated EndoViT: Pretraining Vision Transformers via Federated Learning on Endoscopic Image Collections
- Simplified Swarm Learning Framework for Robust and Scalable Diagnostic Services in Cancer Histopathology
- Private Federated Learning using Preference-Optimized Synthetic Data
- Achieving Distributive Justice in Federated Learning via Uncertainty Quantification
Discussions
Related