Understanding Membership Inferences on Well-Generalized Learning Models

2018/02/13 by Yunhui Long, Vincent Bindschaedler, Long, Yunhui +13 · 17 citations
Computer Science · #Adversarial Robustness in Machine Learning #Cryptography and Security (cs.CR) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Privacy-Preserving Technologies in Data

paper · pdf · doi:10.48550/arxiv.1802.04889

openalex publication_date 2018/02/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Membership Inference Attack (MIA) determines the presence of a record in a machine learning model's training data by querying the model. Prior work has shown that the attack is feasible when the model is overfitted to its training data or when the adversary controls the training algorithm. However, when the model is not overfitted and the adversary does not control the training algorithm, the threat is not well understood. In this paper, we report a study that discovers overfitting to be a sufficient but not a necessary condition for an MIA to succeed. More specifically, we demonstrate that even a well-generalized model contains vulnerable instances subject to a new generalized MIA (GMIA). In GMIA, we use novel techniques for selecting vulnerable instances and detecting their subtle influences ignored by overfitting metrics. Specifically, we successfully identify individual records with high precision in real-world datasets by querying black-box machine learning models. Further we show that a vulnerable record can even be indirectly attacked by querying other related records and existing generalization techniques are found to be less effective in protecting the vulnerable instances. Our findings sharpen the understanding of the fundamental cause of the problem: the unique influences the training instance may have on the model.

Citations

• A Fast Learning Algorithm for Deep Belief Nets
• Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
• Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting

Cited by

• FiMMIA: scaling semantic perturbation-based membership inference across modalities
• Uncovering Pretraining Code in LLMs: A Syntax-Aware Attribution Approach
• Noise Aggregation Analysis Driven by Small-Noise Injection: Efficient Membership Inference for Diffusion Models
• Imitative Membership Inference Attack
• One-Shot Clustering for Federated Learning Under Clustering-Agnostic Assumption
• Privacy Analysis of Deep Learning in the Wild: Membership Inference Attacks against Transfer Learning
• Security Analysis of Deep Neural Networks Operating in the Presence of Cache Side-Channel Attacks
• Privacy for All: Demystify Vulnerability Disparity of Differential Privacy against Membership Inference Attack
• Déjà Vu: an empirical evaluation of the memorization properties of ConvNets
• Evaluating the Dynamics of Membership Privacy in Deep Learning
• Robust Membership Encoding: Inference Attacks and Copyright Protection for Deep Learning
• Cascading and Proxy Membership Inference Attacks
• Measuring Data Leakage in Machine-Learning Models with Fisher Information
• Generative Models for Security: Attacks, Defenses, and Opportunities
• A Comprehensive Analysis of Information Leakage in Deep Transfer Learning
• Is Private Learning Possible with Instance Encoding?
• Securing AI Systems: A Guide to Known Attacks and Impacts

Related

• Membership Inference Attacks Beyond Overfitting
• Fundamental Limits of Membership Inference Attacks on Machine Learning Models
• Data and Model Dependencies of Membership Inference Attack
• Membership Inference Attacks on Machine Learning: A Survey
• On the Vulnerability of Data Points under Multiple Membership Inference Attacks and Target Models