Heterogeneous Gaussian Mechanism: Preserving Differential Privacy in\n Deep Learning with Provable Robustness

2019/06/02 by NhatHai Phan, Phan, NhatHai, Minh T. Vu +11
Computer Science · #Adversarial Robustness in Machine Learning #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.1906.01444

openalex publication_date 2019/06/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this paper, we propose a novel Heterogeneous Gaussian Mechanism (HGM) to\npreserve differential privacy in deep neural networks, with provable robustness\nagainst adversarial examples. We first relax the constraint of the privacy\nbudget in the traditional Gaussian Mechanism from (0, 1] to (0, \∞), with a\nnew bound of the noise scale to preserve differential privacy. The noise in our\nmechanism can be arbitrarily redistributed, offering a distinctive ability to\naddress the trade-off between model utility and privacy loss. To derive\nprovable robustness, our HGM is applied to inject Gaussian noise into the first\nhidden layer. Then, a tighter robustness bound is proposed. Theoretical\nanalysis and thorough evaluations show that our mechanism notably improves the\nrobustness of differentially private deep neural networks, compared with\nbaseline approaches, under a variety of model attacks.\n

Related

• Scalable Differential Privacy with Certified Robustness in Adversarial Learning
• Robustness Threats of Differential Privacy
• Gradient Masking and the Underestimated Robustness Threats of Differential Privacy in Deep Learning
• Adaptive Laplace Mechanism: Differential Privacy Preservation in Deep Learning
• Beyond Laplace and Gaussian: Exploring the Generalized Gaussian Mechanism for Private Machine Learning