The Skellam Mechanism for Differentially Private Federated Learning

2021/10/11 by Naman Agarwal, Peter Kairouz, Agarwal, Naman +3 · 4 citations
Computer Science · #Cryptography and Data Security #Cryptography and Security (cs.CR) #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Mobile Crowdsensing and Crowdsourcing #Privacy-Preserving Technologies in Data #Probability (math.PR)

paper · pdf · doi:10.48550/arxiv.2110.04995

openalex publication_date 2021/10/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce the multi-dimensional Skellam mechanism, a discrete differential privacy mechanism based on the difference of two independent Poisson random variables. To quantify its privacy guarantees, we analyze the privacy loss distribution via a numerical evaluation and provide a sharp bound on the Rényi divergence between two shifted Skellam distributions. While useful in both centralized and distributed privacy applications, we investigate how it can be applied in the context of federated learning with secure aggregation under communication constraints. Our theoretical findings and extensive experimental evaluations demonstrate that the Skellam mechanism provides the same privacy-accuracy trade-offs as the continuous Gaussian mechanism, even when the precision is low. More importantly, Skellam is closed under summation and sampling from it only requires sampling from a Poisson distribution -- an efficient routine that ships with all machine learning and data analysis software packages. These features, along with its discrete nature and competitive privacy-accuracy trade-offs, make it an attractive practical alternative to the newly introduced discrete Gaussian mechanism.

Citations

• LEAF: A Benchmark for Federated Settings
• Differentially Private Learning with Adaptive Clipping
• Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising
• Rényi Differential Privacy of the Sampled Gaussian Mechanism
• Advances and Open Problems in Federated Learning
• Practical and Private (Deep) Learning without Sampling or Shuffling

Cited by

• Infinitely divisible privacy and beyond I: resolution of the s²=2k conjecture
• Federated Computation of ROC and PR Curves
• Securing Private Federated Learning in a Malicious Setting: A Scalable TEE-Based Approach with Client Auditing
• Personalized Subgraph Federated Learning with Sheaf Collaboration

Related

• Skellam Mixture Mechanism: a Novel Approach to Federated Learning with Differential Privacy
• The Poisson binomial mechanism for secure and private federated learning
• Universally Harmonizing Differential Privacy Mechanisms for Federated Learning: Boosting Accuracy and Convergence
• CorBin-FL: A Differentially Private Federated Learning Mechanism using Common Randomness
• Differentially private cross-silo federated learning