Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning

2018/12/31 by Milad Nasr, Reza Shokri, Amir Houmansadr · 4 citations
Computer Science · Mathematics · #Adversarial Robustness in Machine Learning #Adversary #Artificial intelligence #Artificial neural network #Black box #Computer science #Computer security #Data mining #Deep belief network #Deep learning #Inference #Machine learning #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques #White box #cs.CR #cs.LG #stat.ML

paper · pdf · doi:10.1109/sp.2019.00065

2019 IEEE Symposium on Security and Privacy (SP)

openalex publication_date 2019/05/01 · arxiv created 2020/06/06 · arxiv updated 2020/06/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

Deep neural networks are susceptible to various inference attacks as they remember information about their training data. We design white-box inference attacks to perform a comprehensive privacy analysis of deep learning models. We measure the privacy leakage through parameters of fully trained models as well as the parameter updates of models during training. We design inference algorithms for both centralized and federated learning, with respect to passive and active inference attackers, and assuming different adversary prior knowledge. We evaluate our novel white-box membership inference attacks against deep learning algorithms to trace their training data records. We show that a straightforward extension of the known black-box attacks to the white-box setting (through analyzing the outputs of activation functions) is ineffective. We therefore design new algorithms tailored to the white-box setting by exploiting the privacy vulnerabilities of the stochastic gradient descent algorithm, which is the algorithm used to train deep neural networks. We investigate the reasons why deep learning models may leak information about their training data. We then show that even well-generalized models are significantly susceptible to white-box membership inference attacks, by analyzing state-of-the-art pre-trained and publicly available models for the CIFAR dataset. We also show how adversarial participants, in the federated learning setting, can successfully run active membership inference attacks against other participants, even when the global model achieves high prediction accuracies.

Citations

• Calibrating Noise to Sensitivity in Private Data Analysis
• Revealing information while preserving privacy
• A tutorial on spectral clustering
• ImageNet classification with deep convolutional neural networks
• Federated Learning: Strategies for Improving Communication Efficiency
• Scalable Private Learning with PATE
• Understanding Membership Inferences on Well-Generalized Learning Models
• Stealing Hyperparameters in Machine Learning
• Generative Adversarial Privacy
• Understanding deep learning requires rethinking generalization
• Differentially Private Empirical Risk Minimization
• Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
• Learning Multiple Layers of Features from Tiny Images
• The Algorithmic Foundations of Differential Privacy

Cited by

• GRANITE : a Byzantine-Resilient Dynamic Gossip Learning Framework
• When Do PEFT Adaptations Leak Structure? Measuring Black-Box Structural Bounds in Public-Base Model Services
• Towards Communication-Efficient and Attack-Resistant Federated Edge Learning for Industrial Internet of Things
• TrojanDam: Detection-Free Backdoor Defense in Federated Learning through Proactive Model Robustification utilizing OOD Data