Membership Inference Attacks against Machine Learning Models
2016/10/18 by Reza Shokri, Marco Stronati, Shokri, Reza +5 · 211 citations
Computer Science · #Adversarial Robustness in Machine Learning #Cryptography and Security (cs.CR) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Privacy-Preserving Technologies in Data
paper · pdf · doi:10.48550/arxiv.1610.05820
openalex publication_date 2016/10/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Abstract
We quantitatively investigate how machine learning models leak information about the individual data records on which they were trained. We focus on the basic membership inference attack: given a data record and black-box access to a model, determine if the record was in the model's training dataset. To perform membership inference against a target model, we make adversarial use of machine learning and train our own inference model to recognize differences in the target model's predictions on the inputs that it trained on versus the inputs that it did not train on. We empirically evaluate our inference techniques on classification models trained by commercial "machine learning as a service" providers such as Google and Amazon. Using realistic datasets and classification tasks, including a hospital discharge dataset whose membership is sensitive from the privacy perspective, we show that these models can be vulnerable to membership inference attacks. We then investigate the factors that influence this leakage and evaluate mitigation strategies.
Cited by
- FairGFL: Privacy-Preserving Fairness-Aware Federated Learning with Overlapping Subgraphs
- Certifying the Right to Be Forgotten: Primal-Dual Optimization for Sample and Label Unlearning in Vertical Federated Learning
- Bits and Memories: Measuring Verbatim Extraction Across LLM Quantization
- PANOPTICON: A PII-Based Assemblage of Naturalistic Output Tokens for Investigating Privacy Leakage Within LLM Context Window
- It does what it says on the tin: safe synthetic data from coarsened margins
- Assessing the Effectiveness of Membership Inference on Generative Music
- zkFL-Health: Blockchain-Enabled Zero-Knowledge Federated Learning for Medical AI Privacy
- Defending against adversarial attacks using mixture of experts
- Adversarially Robust Detection of Harmful Online Content: A Computational Design Science Approach
- Perturb Your Data: Paraphrase-Guided Training Data Watermarking
- PrivateXR: Defending Privacy Attacks in Extended Reality Through Explainable AI-Guided Differential Privacy
- Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigation
- Dual-View Inference Attack: Machine Unlearning Amplifies Privacy Exposure
- ContextLeak: Auditing Leakage in Private In-Context Learning Methods
- VICTOR: Dataset Copyright Auditing in Video Recognition Systems
- Bits for Privacy: Evaluating Post-Training Quantization via Membership Inference
- An Efficient Gradient-Based Inference Attack for Federated Learning
- PerProb: Indirectly Evaluating Memorization in Large Language Models
- IntentMiner: Intent Inversion Attack via Tool Call Analysis in the Model Context Protocol
- CTIGuardian: A Few-Shot Framework for Mitigating Privacy Leakage in Fine-Tuned LLMs
- On the Effectiveness of Membership Inference in Targeted Data Extraction from Large Language Models
- PRIVEE: Privacy-Preserving Vertical Federated Learning Against Feature Inference Attacks
- A Privacy-Preserving Cloud Architecture for Distributed Machine Learning at Scale
- FLARE: A Wireless Side-Channel Fingerprinting Attack on Federated Learning
- Membership and Dataset Inference Attacks on Large Audio Generative Models
- Reference Recommendation based Membership Inference Attack against Hybrid-based Recommender Systems
- When Tables Leak: Attacking String Memorization in LLM-Based Tabular Data Generation
- When unlearning is free: leveraging low influence points to reduce computational costs
- Exposing and Defending Membership Leakage in Vulnerability Prediction Models
- A Practical Framework for Evaluating Medical AI Security: Reproducible Assessment of Jailbreaking and Privacy Vulnerabilities Across Clinical Specialties
- M-STAR: Multi-Scale Spatiotemporal Autoregression for Human Mobility Modeling
- Prediction with Expert Advice under Local Differential Privacy
- SUGAR: A Sweeter Spot for Generative Unlearning of Many Identities
- Delete and Retain: Efficient Unlearning for Document Classification
- When Privacy Isn't Synthetic: Hidden Data Leakage in Generative AI Models
- RapidUn: Influence-Driven Parameter Reweighting for Efficient Large Language Model Unlearning
- Efficient Public Verification of Private ML via Regularization
- FiMMIA: scaling semantic perturbation-based membership inference across modalities
- How do we measure privacy in text? A survey of text anonymization metrics
- IslandRun: Privacy-Aware Multi-Objective Orchestration for Distributed AI Inference
- Privacy Preserving Diffusion Models for Mixed-Type Tabular Data Generation
- Teleportation-Based Defenses for Privacy in Approximate Machine Unlearning
- One-Shot Secure Aggregation: A Hybrid Cryptographic Protocol for Private Federated Learning in IoT
- Beyond Membership: Limitations of Add/Remove Adjacency in Differential Privacy
- The Double-Edged Nature of the Rashomon Set for Trustworthy Machine Learning
- Trustless Federated Learning at Edge-Scale: A Compositional Architecture for Decentralized, Verifiable, and Incentive-Aligned Coordination
- Are Neuro-Inspired Multi-Modal Vision-Language Models Resilient to Membership Inference Privacy Leakage?
- Are Large Vision Language Models Truly Grounded in Medical Images? Evidence from Italian Clinical Visual Question Answering
- AttackPilot: Autonomous Inference Attacks Against ML Services With LLM-Based Agents
- Fairness Meets Privacy: Integrating Differential Privacy and Demographic Parity in Multi-class Classification
- Membership Inference Attacks Beyond Overfitting
- Privacy-Preserving Federated Learning from Partial Decryption Verifiable Threshold Multi-Client Functional Encryption
- Multimodal Evaluation of Russian-language Architectures
- How to Train Private Clinical Language Models: A Comparative Study of Privacy-Preserving Pipelines for ICD-9 Coding
- As If We've Met Before: LLMs Exhibit Certainty in Recognizing Seen Files
- Effective Code Membership Inference for Code Completion Models via Adversarial Prompts
- Observational Auditing of Label Privacy
- Black-Box Membership Inference Attack for LVLMs via Prior Knowledge-Calibrated Memory Probing
- AI Bill of Materials and Beyond: Systematizing Security Assurance through the AI Risk Scanning (AIRS) Framework
- BudgetLeak: Membership Inference Attacks on RAG Systems via the Generation Budget Side Channel
- "Power of Words": Stealthy and Adaptive Private Information Elicitation via LLM Communication Strategies
- Blockchain-Integrated Privacy-Preserving Medical Insurance Claim Processing Using Homomorphic Encryption
- Biologically-Informed Hybrid Membership Inference Attacks on Generative Genomic Models
- Uncovering Pretraining Code in LLMs: A Syntax-Aware Attribution Approach
- Non-Rival Data as Rival Products: An Encapsulation-Forging Approach for Data Synthesis
- Private-RAG: Answering Multiple Queries with LLMs while Keeping Your Data Private
- Embedding-Space Data Augmentation to Prevent Membership Inference Attacks in Clinical Time Series Forecasting
- PrivacyCD: Hierarchical Unlearning for Protecting Student Privacy in Cognitive Diagnosis
- P-MIA: A Profiled-Based Membership Inference Attack on Cognitive Diagnosis Models
- Who Evaluates AI's Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluations
- Enhancing Federated Learning Privacy with QUBO
- Secure PAC Learning: Sample-Budget Laws and Quantum Data-Path Admissibility
- Improving Unlearning with Model Updates Probably Aligned with Gradients
- FedOnco-Bench: A Reproducible Benchmark for Privacy-Aware Federated Tumor Segmentation with Synthetic CT Data
- EL-MIA: Quantifying Membership Inference Risks of Sensitive Entities in LLMs
- Detecting Data Contamination in LLMs via In-Context Learning
- RECAP: Reproducing Copyrighted Data from LLMs Training with an Agentic Pipeline
- Noise Aggregation Analysis Driven by Small-Noise Injection: Efficient Membership Inference for Diffusion Models
- A Survey on Unlearning in Large Language Models
- OpenLVLM-MIA: A Controlled Benchmark Revealing the Limits of Membership Inference Attacks on Large Vision-Language Models
- A Comprehensive Evaluation Framework for Synthetic Trip Data Generation in Public Transport
- Causal and noncausal contributions to episodic memory: a computational perspective
- Beyond the Trade-off Curve: Multivariate and Advanced Risk-Utility Maps for Evaluating Anonymized and Synthetic Data
- Differential Privacy as a Perk: Federated Learning over Multiple-Access Fading Channels with a Multi-Antenna Base Station
- PrivacyGuard: A Modular Framework for Privacy Auditing in Machine Learning
- Fast-MIA: Efficient and Scalable Membership Inference for LLMs
- Retracing the Past: LLMs Emit Training Data When They Get Lost
- Towards the Formalization of a Trustworthy AI for Mining Interpretable Models explOiting Sophisticated Algorithms
- Black Box Absorption: LLMs Undermining Innovative Ideas
- The Tail Tells All: Estimating Model-Level Membership Inference Vulnerability Without Reference Models
- Training data membership inference via Gaussian process meta-modeling: a post-hoc analysis approach
- Mitigating Privacy-Utility Trade-off in Decentralized Federated Learning via f-Differential Privacy
- Exploring Membership Inference Vulnerabilities in Clinical Large Language Models
- ALPINE: Closed-Loop Adaptive Privacy Budget Allocation for Mobile Edge Crowdsensing
- The Hidden Cost of Modeling P(X): Vulnerability to Membership Inference Attacks in Generative Text Classifiers
- Membership Inference over Diffusion-models-based Synthetic Tabular Data
- SMOTE and Mirrors: Exposing Privacy Leakage from Synthetic Minority Oversampling
- Toward Efficient Inference Attacks: Shadow Model Sharing via Mixture-of-Experts
- Personal Attribute Leakage in Federated Speech Models
- An Investigation of Memorization Risk in Healthcare Foundation Models
- How to Get Actual Privacy and Utility from Privacy Models: the k-Anonymity and Differential Privacy Families
- CoSPED: Consistent Soft Prompt Targeted Data Extraction and Defense
- Catch-Only-One: Non-Transferable Examples for Model-Specific Authorization
- Quantifying Information Disclosure During Gradient Descent Using Gradient Uniqueness
- SoftAdaClip: A Smooth Clipping Strategy for Fair and Private Model Training
- Early Detection and Reduction of Memorisation for Domain Adaptation and Instruction Tuning
- ImpMIA: Leveraging Implicit Bias for Membership Inference Attack under Realistic Scenarios
- f-INE: A Hypothesis Testing Framework for Estimating Influence under Training Randomness
- Private and Fair Machine Learning: Revisiting the Disparate Impact of Differentially Private SGD
- On the Fairness of Privacy Protection: Measuring and Mitigating the Disparity of Group Privacy Risks for Differentially Private Machine Learning
- Detecting Data Contamination from Reinforcement Learning Post-training for Large Language Models
- Enabling Responsible, Secure and Sustainable Healthcare AI - A Strategic Framework for Clinical and Operational Impact
- Approximate Domain Unlearning for Vision-Language Models
- The Model's Language Matters: A Comparative Privacy Analysis of LLMs
- Empirical Comparison of Membership Inference Attacks in Deep Transfer Learning
- Membership Inference Attacks on Tokenizers of Large Language Models
- (Token-Level) InfoRMIA: Stronger Membership Inference and Memorization Assessment for LLMs
- Data Provenance Auditing of Fine-Tuned Large Language Models with a Text-Preserving Technique
- Detecting Distillation Data from Reasoning Models
- RareGraph-Synth: Knowledge-Guided Diffusion Models for Generating Privacy-Preserving Synthetic Patient Trajectories in Ultra-Rare Diseases
- A global log for medical AI
- CryptOracle: A Modular Framework to Characterize Fully Homomorphic Encryption
- External Data Extraction Attacks against Retrieval-Augmented Large Language Models
- Leave No TRACE: Black-box Detection of Copyrighted Dataset Usage in Large Language Models via Watermarking
- SoK: Systematic analysis of adversarial threats against deep learning approaches for autonomous anomaly detection systems in SDN-IoT networks
- DeepProv: Behavioral Characterization and Repair of Neural Networks via Inference Provenance Graph Analysis
- Enhancing Split Learning with Sharded and Blockchain-Enabled SplitFed Approaches
- Score-based Membership Inference on Diffusion Models
- Preserving Cross-Modal Stability for Visual Unlearning in Multimodal Scenarios
- Train Once, Answer All: Many Pretraining Experiments for the Cost of One
- Dual-Space Smoothness for Robust and Balanced LLM Unlearning
- Adaptive Token-Weighted Differential Privacy for LLMs: Not All Tokens Require Equal Protection
- SoK: Potentials and Challenges of Large Language Models for Reverse Engineering
- PQFed: A Privacy-Preserving Quality-Controlled Federated Learning Framework
- Zero-Shot Privacy-Aware Text Rewriting via Iterative Tree Search
- Responsible Diffusion: A Comprehensive Survey on Safety, Ethics, and Trust in Diffusion Models
- From moral panic to pragmatic governance: reframing AI’s societal impacts in employment, education, and ethics
- Enhancing the Effectiveness and Durability of Backdoor Attacks in Federated Learning through Maximizing Task Distinction
- Rethinking Federated Learning Over the Air: The Blessing of Scaling Up
- Attacking LLMs and AI Agents: Advertisement Embedding Attacks Against Large Language Models
- VoxGuard: Evaluating User and Attribute Privacy in Speech via Membership Inference Attacks
- Synth-MIA: A Testbed for Auditing Privacy Leakage in Tabular Data Synthesis
- SEQR: Secure and Efficient QR-based LoRA Routing
- Train to Defend: First Defense Against Cryptanalytic Neural Network Parameter Extraction Attacks
- Causal Fuzzing for Verifying Machine Unlearning
- ToFU: Transforming How Federated Learning Systems Forget User Data
- Privacy Preserving In-Context-Learning Framework for Large Language Models
- MIA-EPT: Membership Inference Attack via Error Prediction for Tabular Data
- Beyond Data Privacy: New Privacy Risks for Large Language Models
- Membership Inference Attack against Large Language Model-based Recommendation Systems: A New Distillation-based Paradigm
- The Sum Leaks More Than Its Parts: Compositional Privacy Risks and Mitigations in Multi-Agent Collaboration
- Sy-FAR: Symmetry-based Fair Adversarial Robustness
- Exposing Privacy Risks in Graph Retrieval-Augmented Generation
- Practitioners' Perspectives on a Differential Privacy Deployment Registry
- Efficient Privacy-Preserving Training of Quantum Neural Networks by Using Mixed States to Represent Input Data Ensembles
- From Firewalls to Frontiers: AI Red-Teaming is a Domain-Specific Evolution of Cyber Red-Teaming
- Membership Inference Attacks on Recommender System: A Survey
- Public Data Assisted Differentially Private In-Context Learning
- LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systems
- Accurate and Private Diagnosis of Rare Genetic Syndromes from Facial Images with Federated Deep Learning
- Differentially Private Decentralized Dataset Synthesis Through Randomized Mixing with Correlated Noise
- A Discrepancy-Based Perspective on Dataset Condensation
- Tight Privacy Audit in One Run
- Active Membership Inference Test (aMINT): Enhancing Model Auditability with Multi-Task Learning
- Uncovering Scaling Laws for Large Language Models via Inverse Problems
- Imitative Membership Inference Attack
- Private Queries with Sigma-Counting
- Preventing Another Tessa: Modular Safety Middleware For Health-Adjacent AI Assistants
- From Membership-Privacy Leakage to Quantum Machine Unlearning
- Graph Unlearning: Efficient Node Removal in Graph Neural Networks
- Privacy Risks in Time Series Forecasting: User- and Record-Level Membership Inference
- A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models
- Variational Gaussian Mixture Manifold Models for Client-Specific Federated Personalization
- On the MIA Vulnerability Gap Between Private GANs and Diffusion Models
- Stealth by Conformity: Evading Robust Aggregation through Adaptive Poisoning
- Privacy-Utility Trade-off in Data Publication: A Bilevel Optimization Framework with Curvature-Guided Perturbation
- Gaming and Cooperation in Federated Learning: What Can Happen and How to Monitor It
- RepoMark: A Data-Usage Auditing Framework for Code Large Language Models
- Privacy Auditing Synthetic Data Release through Local Likelihood Attacks
- AgentCoMa: A Compositional Benchmark Mixing Commonsense and Mathematical Reasoning in Real-World Scenarios
- From Research to Reality: Feasibility of Gradient Inversion Attacks in Federated Learning
- Data Cartography for Detecting Memorization Hotspots and Guiding Data Interventions in Generative Models
- Memorization in Graph Neural Networks
- Tackling Federated Unlearning as a Parameter Estimation Problem
- Towards Reliable and Generalizable Differentially Private Machine Learning (Extended Version)
- BadFU: Backdoor Federated Learning through Adversarial Machine Unlearning
- Linkage Attacks Expose Identity Risks in Public ECG Data Sharing
- On the Security and Privacy of Federated Learning: A Survey with Attacks, Defenses, Frameworks, Applications, and Future Directions
- Unlearning Comparator: A Visual Analytics System for Comparative Evaluation of Machine Unlearning Methods
- Unlearning at Scale: Implementing the Right to be Forgotten in Large Language Models
- Adversarial Robustness in Distributed Quantum Machine Learning
- Assessing User Privacy Leakage in Synthetic Packet Traces: An Attack-Grounded Approach
- SoK: Data Minimization in Machine Learning
- The Surprising Effectiveness of Membership Inference with Simple N-Gram Coverage
- AI Security Map: Holistic Organization of AI Security Technologies and Impacts on Stakeholders
- Assessing and Mitigating Data Memorization Risks in Fine-Tuned Large Language Models
- Membership Inference Attacks with False Discovery Rate Control
- Towards Useful and Private Synthetic Omics: Community Benchmarking of Generative Models for Transcriptomics Data
- FedMeNF: Privacy-Preserving Federated Meta-Learning for Neural Fields
- Membership Inference Attack with Partial Features
- SelectiveShield: Lightweight Hybrid Defense Against Gradient Leakage in Federated Learning
- DP-DocLDM: Differentially Private Document Image Generation using Latent Diffusion Models
- Slice or the Whole Pie? Utility Control for AI Models
- Guess or Recall? Training CNNs to Classify and Localize Memorization in LLMs
- LetheViT: Selective Machine Unlearning for Vision Transformers via Attention-Guided Contrastive Learning
- Win-k: Improved Membership Inference Attacks on Small Language Models
- Provably Secure Retrieval-Augmented Generation
- FedGuard: A Diverse-Byzantine-Robust Mechanism for Federated Learning with Major Malicious Clients
- DICOM De-Identification via Hybrid AI and Rule-Based Framework for Scalable, Uncertainty-Aware Redaction
- Evaluating the Dynamics of Membership Privacy in Deep Learning
- Efficient Machine Unlearning via Influence Approximation
Related