Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning

2022/02/10 by Alberto Bietti, Bietti, Alberto, Chen-Yu Wei +7 · 1 citation
Computer Science · Mathematics · #Advanced Causal Inference Techniques #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2202.05318

openalex publication_date 2022/02/10 · openalex created_date 2022/07/21 · openalex updated_date 2026/07/28

Abstract

Large-scale machine learning systems often involve data distributed across a collection of users. Federated learning algorithms leverage this structure by communicating model updates to a central server, rather than entire datasets. In this paper, we study stochastic optimization algorithms for a personalized federated learning setting involving local and global models subject to user-level (joint) differential privacy. While learning a private global model induces a cost of privacy, local learning is perfectly private. We provide generalization guarantees showing that coordinating local learning with private centralized learning yields a generically useful and improved tradeoff between accuracy and privacy. We illustrate our theoretical results with experiments on synthetic and real-world datasets.

Cited by

• ADP-VRSGP: Decentralized Learning with Adaptive Differential Privacy via Variance-Reduced Stochastic Gradient Push

Related

• Federated Learning with Differential Privacy: An Utility-Enhanced Approach
• Survey of Personalization Techniques for Federated Learning
• LDP-FL: Practical Private Aggregation in Federated Learning with Local Differential Privacy
• Concentrated Differentially Private and Utility Preserving Federated Learning
• Federated Learning with Enhanced Privacy via Model Splitting and Random Client Participation