Differentially Private Generation of Small Images

2020/05/02 by Justus T. C. Schwabedal, Schwabedal, Justus T. C., Pascal Michel +3
Computer Science · #Adversarial Robustness in Machine Learning #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2005.00783

openalex publication_date 2020/05/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We explore the training of generative adversarial networks with differential privacy to anonymize image data sets. On MNIST, we numerically measure the privacy-utility trade-off using parameters from ε-δ differential privacy and the inception score. Our experiments uncover a saturated training regime where an increasing privacy budget adds little to the quality of generated images. We also explain analytically why differentially private Adam optimization is independent of the gradient clipping parameter. Furthermore, we highlight common errors in previous works on differentially private deep learning, which we uncovered in recent literature. Throughout the treatment of the subject, we hope to prevent erroneous estimates of anonymity in the future.

Citations

• Rényi Differential Privacy of the Sampled Gaussian Mechanism
• A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications

Related

• DP-CGAN: Differentially Private Synthetic Data and Label Generation
• Deep Learning with Gaussian Differential Privacy
• PATE-TripleGAN: Privacy-Preserving Image Synthesis with Gaussian Differential Privacy
• Differentially Private Generative Adversarial Network
• Large-Scale Public Data Improves Differentially Private Image Generation Quality