Towards Federated Learning at Scale: System Design
2019/02/04 by Keith Bonawitz, Bonawitz, Keith, Hubert Eichner +26 · 2 voices · 216 citations
Computer Science · Mathematics · #Advanced Graph Neural Networks #Mobile Crowdsensing and Crowdsourcing #Privacy-Preserving Technologies in Data #cs.DC #cs.LG #stat.ML
paper · pdf · doi:10.48550/arxiv.1902.01046
arxiv created 2019/03/22 · arxiv updated 2019/03/26
Abstract
Federated Learning is a distributed machine learning approach which enables model training on a large corpus of decentralized data. We have built a scalable production system for Federated Learning in the domain of mobile devices, based on TensorFlow. In this paper, we describe the resulting high-level design, sketch some of the challenges and their solutions, and touch upon the open problems and future directions.
Citations
Cited by
- Adaptive Data Admission and Retention for Streaming Federated Learning
- Learned Digital Codes for Over-the-Air Computation in Federated Edge Learning
- FedWiLoc: Federated Learning for Privacy-Preserving WiFi Indoor Localization
- Practical Framework for Privacy-Preserving and Byzantine-robust Federated Learning
- Distributed HDMM: Scalable, Distributed, Accurate, and Differentially Private Query Workloads without a Trusted Curator
- LCMem: A Universal Model for Robust Image Memorization Detection
- Geometric Prior-Guided Federated Prompt Calibration
- Breaking the Capacity Bottleneck in Model-Heterogeneous Federated Learning via Gradual Model Restoration
- Multi-Frequency Federated Learning for Human Activity Recognition Using Head-Worn Sensors
- Federated style aware transformer aggregation of representations
- TOFA: Training-Free One-Shot Federated Adaptation for Vision-Language Models
- A Hybrid Proactive And Predictive Framework For Edge Cloud Resource Management
- Analyzing the Impact of Participant Failures in Cross-Silo Federated Learning
- A Unified Convergence Analysis for Semi-Decentralized Learning: Sampled-to-Sampled vs. Sampled-to-All Communication
- SMoFi: Step-wise Momentum Fusion for Split Federated Learning on Heterogeneous Data
- AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning
- Sustainable AI: Environmental Implications, Challenges and Opportunities
- An Efficient Gradient-Aware Error-Bounded Lossy Compressor for Federated Learning
- Edge AI in Highly Volatile Environments: Is Fairness Worth the Accuracy Trade-off?
- Reviving Stale Updates: Data-Free Knowledge Distillation for Asynchronous Federated Learning
- FedMGP: Personalized Federated Learning with Multi-Group Text-Visual Prompts
- Privacy-Preserving Multi-Party Contextual Bandits
- Flower: A Friendly Federated Learning Research Framework
- Hierarchical Federated Learning through LAN-WAN Orchestration
- Federated Mixture of Experts
- Prophet: Proactive Candidate-Selection for Federated Learning by Predicting the Qualities of Training and Reporting Phases
- ABC-FL: Anomalous and Benign client Classification in Federated Learning
- Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications
- FAT: Federated Adversarial Training
- SECure: A Social and Environmental Certificate for AI Systems
- Performance Analysis and Characterization of Training Deep Learning Models on Mobile Devices
- Bayesian Federated Learning over Wireless Networks
- Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus
- Federated Learning with Buffered Asynchronous Aggregation
- Energy Efficient Federated Learning Over Wireless Communication Networks
- Federated Learning for Smart Healthcare: A Survey
- Orthogonal Gradient Descent for Continual Learning
- Robust Aggregation for Federated Learning
- Personalized Federated Learning for Heterogeneous Clients with Clustered Knowledge Transfer
- Mixing Deep Learning and Multiple Criteria Optimization: An Application to Distributed Learning with Multiple Datasets
- Federated Learning for Big Data: A Survey on Opportunities, Applications, and Future Directions
- SemiFL: Semi-Supervised Federated Learning for Unlabeled Clients with Alternate Training
- Separation of Powers in Federated Learning
- WAFFLE: Watermarking in Federated Learning
- Practical and Private (Deep) Learning without Sampling or Shuffling
- A survey on federated learning
- Cluster-Based Client Selection for Dependent Multi-Task Federated Learning in Edge Computing
- Research in Collaborative Learning Does Not Serve Cross-Silo Federated Learning in Practice
- PubSub-VFL: Towards Efficient Two-Party Split Learning in Heterogeneous Environments via Publisher/Subscriber Architecture
- FedHybrid: Breaking the Memory Wall of Federated Learning via Hybrid Tensor Management
- Decoupled DiLoCo for Resilient Distributed Pre-training
- Applications of Federated Learning in Smart Cities: Recent Advances, Taxonomy, and Open Challenges
- Evaluation of Differential Privacy Mechanisms on Federated Learning
- Small-Group Learning, with Application to Neural Architecture Search
- Multi-Task Federated Reinforcement Learning with Adversaries
- Parameter-Free Federated TD Learning with Markov Noise in Heterogeneous Environments
- Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
- Beyond Static Knowledge Messengers: Towards Adaptive, Fair, and Scalable Federated Learning for Medical AI
- Asynchronous Federated Learning with Reduced Number of Rounds and with Differential Privacy from Less Aggregated Gaussian Noise
- Personalized Federated Learning with First Order Model Optimization
- Towards Utilizing Unlabeled Data in Federated Learning: A Survey and Prospective
- Characterizing the Performance of Accelerated Jetson Edge Devices for Training Deep Learning Models
- Towards Adapting Federated & Quantum Machine Learning for Network Intrusion Detection: A Survey
- Probabilistic Federated Learning of Neural Networks Incorporated with Global Posterior Information
- Curse or Redemption? How Data Heterogeneity Affects the Robustness of Federated Learning
- EigenGame: PCA as a Nash Equilibrium
- Optimizing Resource-Efficiency for Federated Edge Intelligence in IoT Networks
- OmniFed: A Modular Framework for Configurable Federated Learning from Edge to HPC
- Faster On-Device Training Using New Federated Momentum Algorithm
- Ensemble Distillation for Robust Model Fusion in Federated Learning
- FedAVOT: Exact Distribution Alignment in Federated Learning via Masked Optimal Transport
- Proof of Federated Learning: A Novel Energy-recycling Consensus Algorithm
- Federated Meta-Learning with Fast Convergence and Efficient Communication
- Federated Edge Learning for Predictive Maintenance in 6G Small Cell Networks
- Adversarial Examples Make Strong Poisons
- PracMHBench: Re-evaluating Model-Heterogeneous Federated Learning Based on Practical Edge Device Constraints
- ResiliNet: Failure-Resilient Inference in Distributed Neural Networks
- Scheduling for Cellular Federated Edge Learning with Importance and Channel Awareness
- FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tiers
- SAPAG: A Self-Adaptive Privacy Attack From Gradients
- Federated Learning via Posterior Averaging: A New Perspective and Practical Algorithms
- Deep Leakage from Gradients
- SmartFLow: A Communication-Efficient SDN Framework for Cross-Silo Federated Learning
- FedV: Privacy-Preserving Federated Learning over Vertically Partitioned Data
- FedPrune: Towards Inclusive Federated Learning
- Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Review
- Coded Computing for Master-Aided Distributed Computing Systems
- Decentralized Weather Forecasting via Distributed Machine Learning and Blockchain-Based Model Validation
- Edge Intelligence: Architectures, Challenges, and Applications
- Optimizing Federated Learning for Scalable Power-demand Forecasting in Microgrids
- FuSeFL: Fully Secure and Scalable Federated Learning
- 6G White paper: Research challenges for Trust, Security and Privacy
- Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges
- SenseCrypt: Sensitivity-guided Selective Homomorphic Encryption for Joint Federated Learning in Cross-Device Scenarios
- Training Production Language Models without Memorizing User Data
- Dynamic Clustering for Personalized Federated Learning on Heterogeneous Edge Devices
- Asynchronous Federated Learning with non-convex client objective functions and heterogeneous dataset
- Multi-task Federated Edge Learning (MtFEEL) in Wireless Networks
- Towards Ubiquitous AI in 6G with Federated Learning
- FedCVD++: Communication-Efficient Federated Learning for Cardiovascular Risk Prediction with Parametric and Non-Parametric Model Optimization
- FedABC: Attention-Based Client Selection for Federated Learning with Long-Term View
- Abnormal Client Behavior Detection in Federated Learning
- Canoe : A System for Collaborative Learning for Neural Nets
- Revocable Federated Learning: A Benchmark of Federated Forest
- PowerTrip: Exploiting Federated Heterogeneous Datacenter Power for Distributed ML Training
- Efficient and Private Federated Learning with Partially Trainable Networks
- A Vertical Federated Learning Framework for Graph Convolutional Network
- LDP-Fed: Federated Learning with Local Differential Privacy
- Accelerating Federated Learning in Heterogeneous Data and Computational Environments
- Budgeted Online Selection of Candidate IoT Clients to Participate in Federated Learning
- Federated Echo State Learning for Minimizing Breaks in Presence in Wireless Virtual Reality Networks
- MDLdroid: a ChainSGD-reduce Approach to Mobile Deep Learning for Personal Mobile Sensing
- An Exploratory Analysis on Users' Contributions in Federated Learning
- Federated Dynamic GNN with Secure Aggregation
- FedChip: Federated LLM for Artificial Intelligence Accelerator Chip Design
- Preventing Adversarial Use of Datasets through Fair Core-Set Construction
- Adaptive Distillation for Decentralized Learning from Heterogeneous Clients
- Enabling Design Methodologies and Future Trends for Edge AI: Specialization and Co-design
- Neurosciences and 6G: Lessons from and Needs of Communicative Brains
- Slashing Communication Traffic in Federated Learning by Transmitting Clustered Model Updates
- Federated Edge Learning with Misaligned Over-The-Air Computation
- Papaya: Practical, Private, and Scalable Federated Learning
- A Survey on Large-scale Machine Learning
- Privacy Preservation in Federated Learning: An insightful survey from the GDPR Perspective
- Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge
- Dropout against Deep Leakage from Gradients
- Efficient Federated Learning with Heterogeneous Data and Adaptive Dropout
- Convergence of Agnostic Federated Averaging
- Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learning
- IBM Federated Learning: an Enterprise Framework White Paper V0.1
- Federated Classification using Parsimonious Functions in Reproducing Kernel Hilbert Spaces
- OpenFed: A Comprehensive and Versatile Open-Source Federated Learning Framework
- Efficient Federated Learning with Timely Update Dissemination
- TiFL: A Tier-based Federated Learning System
- Loss Tolerant Federated Learning
- FedQML-Edge: Compact Quantum Feature Sketches for Communication-Constrained Roadside Federated Learning
- A Joint Energy and Latency Framework for Transfer Learning over 5G Industrial Edge Networks
- Practical Defences Against Model Inversion Attacks for Split Neural Networks
- Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning
- Federated Noisy Client Learning
- CLoVE: Personalized Federated Learning through Clustering of Loss Vector Embeddings
- Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients
- Topology-Aware Differential Privacy in Federated Learning
- BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learning
- Byzantine-resilient Decentralized Stochastic Gradient Descent
- SecureFed: A Two-Phase Framework for Detecting Malicious Clients in Federated Learning
- Clustered Federated Learning via Embedding Distributions
- A Federated Random Forest Solution for Secure Distributed Machine Learning
- All One Needs to Know about Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agenda
- Wavelet Scattering Transform and Fourier Representation for Offline Detection of Malicious Clients in Federated Learning
- FedAPM: Federated Learning via ADMM with Partial Model Personalization
- FedSel: Federated SGD under Local Differential Privacy with Top-k Dimension Selection
- Distributed Training with Heterogeneous Data: Bridging Median- and Mean-Based Algorithms
- Hogwild! over Distributed Local Data Sets with Linearly Increasing Mini-Batch Sizes
- Consensus Driven Learning
- FedLoc: Federated Learning Framework for Data-Driven Cooperative Localization and Location Data Processing
- Pronto: Federated Task Scheduling
- PC-MoE: Memory-Efficient and Privacy-Preserving Collaborative Training for Mixture-of-Experts LLMs
- Acceleration of Cooperative Least Mean Square via Chebyshev Periodical Successive Over-Relaxation
- MAB-based Client Selection for Federated Learning with Uncertain Resources in Mobile Networks
- Secure Byzantine-Robust Distributed Learning via Clustering
- Small-Scale-Fading-Aware Resource Allocation in Wireless Federated Learning
- ByGARS: Byzantine SGD with Arbitrary Number of Attackers
- FLRA: A Reference Architecture for Federated Learning Systems
- Improving Fairness via Federated Learning
- Adaptive Deadline and Batch Layered Synchronized Federated Learning
- DeCAF: Decentralized Consensus-And-Factorization for Low-Rank Adaptation of Foundation Models
- Unified Alignment Protocol: Making Sense of the Unlabeled Data in New Domains
- Fairness in Federated Learning: Fairness for Whom?
- Label Leakage in Federated Inertial-based Human Activity Recognition
- A Federated Learning Approach for Mobile Packet Classification
- Private Federated Learning with Domain Adaptation
- Addressing Data Quality Decompensation in Federated Learning via Dynamic Client Selection
- Distributed Learning and its Application for Time-Series Prediction
- FedGEMS: Federated Learning of Larger Server Models via Selective Knowledge Fusion
- Corella: A Private Multi Server Learning Approach based on Correlated Queries
- C3DRec: Cloud-Client Cooperative Deep Learning for Temporal Recommendation in the Post-GDPR Era
- Adaptive Federated Optimization
- Communication Efficient Federated Learning with Energy Awareness over Wireless Networks
- Towards Non-I.I.D. and Invisible Data with FedNAS: Federated Deep Learning via Neural Architecture Search
- EdgeAgentX: A Novel Framework for Agentic AI at the Edge in Military Communication Networks
- OL4EL: Online Learning for Edge-cloud Collaborative Learning on Heterogeneous Edges with Resource Constraints
- A Two-Stage Data Selection Framework for Data-Efficient Model Training on Edge Devices
- Federated Learning for Healthcare Informatics
- FedSplit: An algorithmic framework for fast federated optimization
- DEAL: Decremental Energy-Aware Learning in a Federated System
- PyVertical: A Vertical Federated Learning Framework for Multi-headed SplitNN
- RIFLES: Resource-effIcient Federated LEarning via Scheduling
- SecEmb: Sparsity-Aware Secure Federated Learning of On-Device Recommender System with Large Embedding
- A Vertical Federated Learning Framework for Horizontally Partitioned Labels
- Exploiting Heterogeneity in Robust Federated Best-Arm Identification
- Management of Resource at the Network Edge for Federated Learning
- Preserving Patient Privacy while Training a Predictive Model of In-hospital Mortality
- Toward Malicious Clients Detection in Federated Learning
- Energy-Efficient Federated Learning for AIoT using Clustering Methods
- FGLP: A Federated Fine-Grained Location Prediction System for Mobile Users
- PQS-BFL: A Post-Quantum Secure Blockchain-based Federated Learning Framework
- Towards Scheduling Federated Deep Learning using Meta-Gradients for Inter-Hospital Learning
- Prior-Independent Auctions for the Demand Side of Federated Learning
- Empowering clinical research in a decentralized world
- Real-World Image Datasets for Federated Learning
- Multi-Message Secure Aggregation with Demand Privacy
- Federated Learning: Opportunities and Challenges
- Federated Block Coordinate Descent Scheme for Learning Global and Personalized Models
- Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks
- Federated Extra-Trees with Privacy Preserving
- Federated Learning: A Survey on Privacy-Preserving Collaborative Intelligence
- Privacy preserving distributed machine learning with federated learning
- SparsyFed: Sparse Adaptive Federated Training
- Towards a Distributed Federated Learning Aggregation Placement using Particle Swarm Intelligence
- Two-stage Federated Phenotyping and Patient Representation Learning
- UVeQFed: Universal Vector Quantization for Federated Learning
- FedFetch: Faster Federated Learning with Adaptive Downstream Prefetching
- OnSlicing
- Learning, Computing, and Trustworthiness in Intelligent IoT Environments: Performance-Energy Tradeoffs
- Collaborative Learning of On-Device Small Model and Cloud-Based Large Model: Advances and Future Directions
- Fast-adapting and Privacy-preserving Federated Recommender System
- BoTTA: Benchmarking on-device Test Time Adaptation
- Federated Unlearning Made Practical: Seamless Integration via Negated Pseudo-Gradients
Discussions
Related