Subsampled R 'enyi Differential Privacy and Analytical Moments\n Accountant

2018/07/31 by Yu-Xiang Wang, Borja Balle, Wang, Yu-Xiang +3 · 21 citations
Computer Science · Decision Sciences · #Cryptography and Data Security #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Privacy-Preserving Technologies in Data #Probability and Risk Models

paper · pdf · doi:10.48550/arxiv.1808.00087

openalex publication_date 2018/07/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We study the problem of subsampling in differential privacy (DP), a question\nthat is the centerpiece behind many successful differentially private machine\nlearning algorithms. Specifically, we provide a tight upper bound on the\nR 'enyi Differential Privacy (RDP) (Mironov, 2017) parameters for algorithms\nthat: (1) subsample the dataset, and then (2) applies a randomized mechanism M\nto the subsample, in terms of the RDP parameters of M and the subsampling\nprobability parameter. Our results generalize the moments accounting technique,\ndeveloped by Abadi et al. (2016) for the Gaussian mechanism, to any subsampled\nRDP mechanism.\n

Cited by

• Composition Theorems for f-Differential Privacy
• Towards Privacy-Preserving Code Generation: Differentially Private Code Language Models
• Infinitely divisible privacy and beyond I: resolution of the s²=2k conjecture
• PrAda-GAN: A Private Adaptive Generative Adversarial Network with Bayes Network Structure
• Private-RAG: Answering Multiple Queries with LLMs while Keeping Your Data Private
• Perturbation Bounds for Low-Rank Inverse Approximations under Noise
• Mitigating Privacy-Utility Trade-off in Decentralized Federated Learning via f-Differential Privacy
• ACTG-ARL: Differentially Private Conditional Text Generation with RL-Boosted Control
• SoftAdaClip: A Smooth Clipping Strategy for Fair and Private Model Training
• An information theorist's tour of differential privacy
• Differentially Private Two-Stage Gradient Descent for Instrumental Variable Regression
• Differential Privacy for Euclidean Jordan Algebra with Applications to Private Symmetric Cone Programming
• Differentially Private Decentralized Dataset Synthesis Through Randomized Mixing with Correlated Noise
• PLRV-O: Advancing Differentially Private Deep Learning via Privacy Loss Random Variable Optimization
• Towards Reliable and Generalizable Differentially Private Machine Learning (Extended Version)
• Beyond Trade-offs: A Unified Framework for Privacy, Robustness, and Communication Efficiency in Federated Learning
• Dynamic Differential-Privacy Preserving SGD
• Private and Communication-Efficient Edge Learning: A Sparse Differential Gaussian-Masking Distributed SGD Approach
• Differentially Private Federated Clustering with Random Rebalancing
• Training Production Language Models without Memorizing User Data
• Improving Noise Efficiency in Privacy-preserving Dataset Distillation

Related

• Rényi Differential Privacy of the Sampled Gaussian Mechanism
• A Better Bound Gives a Hundred Rounds: Enhanced Privacy Guarantees via\n f-Divergences
• Three Variants of Differential Privacy: Lossless Conversion and\n Applications
• Unified Mechanism-Specific Amplification by Subsampling and Group Privacy Amplification
• Beyond Laplace and Gaussian: Exploring the Generalized Gaussian Mechanism for Private Machine Learning