Towards Federated Learning at Scale: System Design
2019/02/04 by Keith Bonawitz, Bonawitz, Keith, Hubert Eichner +26 · 2 voices · 107 citations
Computer Science · #Privacy-Preserving Technologies in Data #Mobile Crowdsensing and Crowdsourcing #Advanced Graph Neural Networks
paper · pdf · doi:10.48550/arxiv.1902.01046
Abstract
Federated Learning is a distributed machine learning approach which enables model training on a large corpus of decentralized data. We have built a scalable production system for Federated Learning in the domain of mobile devices, based on TensorFlow. In this paper, we describe the resulting high-level design, sketch some of the challenges and their solutions, and touch upon the open problems and future directions.
Citations
Cited by
- Adaptive Data Admission and Retention for Streaming Federated Learning
- Learned Digital Codes for Over-the-Air Computation in Federated Edge Learning
- FedWiLoc: Federated Learning for Privacy-Preserving WiFi Indoor Localization
- Practical Framework for Privacy-Preserving and Byzantine-robust Federated Learning
- Distributed HDMM: Scalable, Distributed, Accurate, and Differentially Private Query Workloads without a Trusted Curator
- LCMem: A Universal Model for Robust Image Memorization Detection
- Geometric Prior-Guided Federated Prompt Calibration
- Breaking the Capacity Bottleneck in Model-Heterogeneous Federated Learning via Gradual Model Restoration
- Multi-Frequency Federated Learning for Human Activity Recognition Using Head-Worn Sensors
- Federated style aware transformer aggregation of representations
- TOFA: Training-Free One-Shot Federated Adaptation for Vision-Language Models
- A Hybrid Proactive And Predictive Framework For Edge Cloud Resource Management
- Analyzing the Impact of Participant Failures in Cross-Silo Federated Learning
- A Unified Convergence Analysis for Semi-Decentralized Learning: Sampled-to-Sampled vs. Sampled-to-All Communication
- SMoFi: Step-wise Momentum Fusion for Split Federated Learning on Heterogeneous Data
- AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning
- Sustainable AI: Environmental Implications, Challenges and Opportunities
- An Efficient Gradient-Aware Error-Bounded Lossy Compressor for Federated Learning
- Edge AI in Highly Volatile Environments: Is Fairness Worth the Accuracy Trade-off?
- Reviving Stale Updates: Data-Free Knowledge Distillation for Asynchronous Federated Learning
- FedMGP: Personalized Federated Learning with Multi-Group Text-Visual Prompts
- Privacy-Preserving Multi-Party Contextual Bandits
- Flower: A Friendly Federated Learning Research Framework
- Hierarchical Federated Learning through LAN-WAN Orchestration
- Federated Mixture of Experts
- Prophet: Proactive Candidate-Selection for Federated Learning by Predicting the Qualities of Training and Reporting Phases
- ABC-FL: Anomalous and Benign client Classification in Federated Learning
- Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications
- FAT: Federated Adversarial Training
- SECure: A Social and Environmental Certificate for AI Systems
- Performance Analysis and Characterization of Training Deep Learning Models on Mobile Devices
- Bayesian Federated Learning over Wireless Networks
- Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus
- Federated Learning with Buffered Asynchronous Aggregation
- Energy Efficient Federated Learning Over Wireless Communication Networks
- Federated Learning for Smart Healthcare: A Survey
- Orthogonal Gradient Descent for Continual Learning
- Robust Aggregation for Federated Learning
- Personalized Federated Learning for Heterogeneous Clients with Clustered Knowledge Transfer
- Mixing Deep Learning and Multiple Criteria Optimization: An Application to Distributed Learning with Multiple Datasets
- Federated Learning for Big Data: A Survey on Opportunities, Applications, and Future Directions
- SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training
- Separation of Powers in Federated Learning
- WAFFLE: Watermarking in Federated Learning
- Practical and Private (Deep) Learning without Sampling or Shuffling
- A survey on federated learning
- Cluster-Based Client Selection for Dependent Multi-Task Federated Learning in Edge Computing
- Research in Collaborative Learning Does Not Serve Cross-Silo Federated Learning in Practice
- PubSub-VFL: Towards Efficient Two-Party Split Learning in Heterogeneous Environments via Publisher/Subscriber Architecture
- FedHybrid: Breaking the Memory Wall of Federated Learning via Hybrid Tensor Management
- Decoupled DiLoCo for Resilient Distributed Pre-training
- Applications of Federated Learning in Smart Cities: Recent Advances, Taxonomy, and Open Challenges
- Evaluation of Differential Privacy Mechanisms on Federated Learning
- Small-Group Learning, with Application to Neural Architecture Search
- Multi-Task Federated Reinforcement Learning with Adversaries
- Parameter-Free Federated TD Learning with Markov Noise in Heterogeneous Environments
- Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
- Beyond Static Knowledge Messengers: Towards Adaptive, Fair, and Scalable Federated Learning for Medical AI
- Asynchronous Federated Learning with Reduced Number of Rounds and with Differential Privacy from Less Aggregated Gaussian Noise
- Personalized Federated Learning with First Order Model Optimization
- Towards Utilizing Unlabeled Data in Federated Learning: A Survey and Prospective
- Characterizing the Performance of Accelerated Jetson Edge Devices for Training Deep Learning Models
- Towards Adapting Federated & Quantum Machine Learning for Network Intrusion Detection: A Survey
- Probabilistic Federated Learning of Neural Networks Incorporated with Global Posterior Information
- Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning
- EigenGame: PCA as a Nash Equilibrium
- Optimizing Resource-Efficiency for Federated Edge Intelligence in IoT Networks
- OmniFed: A Modular Framework for Configurable Federated Learning from Edge to HPC
- Faster On-Device Training Using New Federated Momentum Algorithm
- Ensemble Distillation for Robust Model Fusion in Federated Learning
- FedAVOT: Exact Distribution Alignment in Federated Learning via Masked Optimal Transport
- Proof of Federated Learning: A Novel Energy-recycling Consensus Algorithm
- Federated Meta-Learning with Fast Convergence and Efficient Communication
- Federated Edge Learning for Predictive Maintenance in 6G Small Cell Networks
- Adversarial Examples Make Strong Poisons
- PracMHBench: Re-evaluating Model-Heterogeneous Federated Learning Based on Practical Edge Device Constraints
- ResiliNet: Failure-Resilient Inference in Distributed Neural Networks
- Scheduling for Cellular Federated Edge Learning with Importance and Channel Awareness
- FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tiers
- SAPAG: A Self-Adaptive Privacy Attack From Gradients
- Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms
- Deep Leakage from Gradients
- SmartFLow: A Communication-Efficient SDN Framework for Cross-Silo Federated Learning
- FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data
- FedPrune: Towards Inclusive Federated Learning
- Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review
- Coded Computing for Master-Aided Distributed Computing Systems
- Decentralized Weather Forecasting via Distributed Machine Learning and Blockchain-Based Model Validation
- Edge Intelligence: Architectures, Challenges, and Applications
- Optimizing Federated Learning for Scalable Power-demand Forecasting in Microgrids
- SenseCrypt: Sensitivity-guided Selective Homomorphic Encryption for Joint Federated Learning in Cross-Device Scenarios
- Training Production Language Models without Memorizing User Data
- Dynamic Clustering for Personalized Federated Learning on Heterogeneous Edge Devices
- Asynchronous Federated Learning with non-convex client objective functions and heterogeneous dataset
- Multi-task Federated Edge Learning (MtFEEL) in Wireless Networks
- Towards Ubiquitous AI in 6G with Federated Learning
- FedCVD++: Communication-Efficient Federated Learning for Cardiovascular Risk Prediction with Parametric and Non-Parametric Model Optimization
- FedABC: Attention-Based Client Selection for Federated Learning with Long-Term View
- Abnormal Client Behavior Detection in Federated Learning
- Canoe : A System for Collaborative Learning for Neural Nets
- Revocable Federated Learning: A Benchmark of Federated Forest
- PowerTrip: Exploiting Federated Heterogeneous Datacenter Power for Distributed ML Training
- Efficient and Private Federated Learning with Partially Trainable Networks
- A Vertical Federated Learning Framework for Graph Convolutional Network
- LDP-Fed: Federated Learning with Local Differential Privacy
- Accelerating Federated Learning in Heterogeneous Data and Computational\n Environments
- Budgeted Online Selection of Candidate IoT Clients to Participate in\n Federated Learning
- Federated Echo State Learning for Minimizing Breaks in Presence in Wireless Virtual Reality Networks
- MDLdroid: a ChainSGD-reduce Approach to Mobile Deep Learning for Personal Mobile Sensing
Discussions
Related