The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation

2021/02/12 by Peter Kairouz, Ziyu Liu, Kairouz, Peter +3 · 16 citations
Computer Science · #Cryptography and Data Security #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2102.06387

openalex publication_date 2021/02/12 · openalex created_date 2022/09/13 · openalex updated_date 2026/07/28

Abstract

We consider training models on private data that are distributed across user devices. To ensure privacy, we add on-device noise and use secure aggregation so that only the noisy sum is revealed to the server. We present a comprehensive end-to-end system, which appropriately discretizes the data and adds discrete Gaussian noise before performing secure aggregation. We provide a novel privacy analysis for sums of discrete Gaussians and carefully analyze the effects of data quantization and modular summation arithmetic. Our theoretical guarantees highlight the complex tension between communication, privacy, and accuracy. Our extensive experimental results demonstrate that our solution is essentially able to match the accuracy to central differential privacy with less than 16 bits of precision per value.

Cited by

• Pruning as a Game: Equilibrium-Driven Sparsification of Neural Networks
• Distributed HDMM: Scalable, Distributed, Accurate, and Differentially Private Query Workloads without a Trusted Curator
• Bayesian Coreset Optimization for Personalized Federated Learning
• DP-FedPGN: Finding Global Flat Minima for Differentially Private Federated Learning via Penalizing Gradient Norm
• Toward provably private analytics and insights into GenAI use
• DP-HYPE: Distributed Differentially Private Hyperparameter Search
• Federated Computation of ROC and PR Curves
• GPM: The Gaussian Pancake Mechanism for Planting Undetectable Backdoors in Differential Privacy
• Securing Private Federated Learning in a Malicious Setting: A Scalable TEE-Based Approach with Client Auditing
• One-Bit Model Aggregation for Differentially Private and Byzantine-Robust Personalized Federated Learning
• Privacy-Preserving Quantized Federated Learning with Diverse Precision
• Byzantine Outside, Curious Inside: Reconstructing Data Through Malicious Updates
• FedAPM: Federated Learning via ADMM with Partial Model Personalization
• Dropout-Robust Mechanisms for Differentially Private and Fully Decentralized Mean Estimation
• Differential Privacy Analysis of Decentralized Gossip Averaging under Varying Threat Models
• VDDP: Verifiable Distributed Differential Privacy under the Client-Server-Verifier Setup

Related

• Differentially Private Federated Learning without Noise Addition: When is it Possible?
• On Using Secure Aggregation in Differentially Private Federated Learning with Multiple Local Steps
• Perfectly-Private Analog Secure Aggregation in Federated Learning
• The Fundamental Price of Secure Aggregation in Differentially Private Federated Learning
• Efficient Differentially Private Secure Aggregation for Federated Learning via Hardness of Learning with Errors