A Unified Framework for Adversary-Aware Differential Privacy Bounds

2025/07/10 by Marika Swanberg, Swanberg, Marika, Meenatchi Sundaram Muthu Selva Annamalai +7 · 1 citation
Computer Science · Engineering · #Adversarial Robustness in Machine Learning #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Privacy-Preserving Technologies in Data #Smart Grid Security and Resilience

paper · pdf · doi:10.48550/arxiv.2507.08158

openalex publication_date 2025/07/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Differential Privacy (DP) bounds the privacy leakage of a mechanism against worst-case membership inference, but the precise tradeoff between complex adversarial models and DP protections remains poorly understood. In this paper, we present a unified framework that generalizes the patchwork of existing bounds across membership inference, attribute inference, and data reconstruction attacks. Crucially, our framework is the first to evaluate attacks that target multiple individuals simultaneously and measure success beyond exact matches under a single cohesive bound. Our bounds capture this broad family of previously unexplored attack settings by relying solely on the privacy parameters and the adversary's baseline success rate (i.e. its prior without access to the mechanism's output). To illustrate this, we compare our high-probability guarantees to empirical attacks in two novel settings: extracting multiple non-uniform secrets (passwords and PII) from DP-finetuned language models, and reconstructing tabular data from noisy marginals. Ultimately, this framework provides a rigorous theoretical foundation to investigate the risk landscape of DP algorithms in new adversarial settings.

Citations

• VaultGemma: A Differentially Private Gemma Model
• Unifying Re-Identification, Attribute Inference, and Data Reconstruction Risks in Differential Privacy
• Language Models May Verbatim Complete Text They Were Not Explicitly Trained On
• How Well Can Differential Privacy Be Audited in One Run?
• Privacy Auditing of Large Language Models
• Auditing f-Differential Privacy in One Run
• The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD
• Data Reconstruction: When You See It and When You Don't
• Tighter Privacy Auditing of DP-SGD in the Hidden State Threat Model
• Nearly Tight Black-Box Auditing of Differentially Private Machine Learning
• ATTAXONOMY: Unpacking Differential Privacy Guarantees Against Practical Adversaries
• Privacy Auditing with One (1) Training Run
• Bounding Training Data Reconstruction in DP-SGD
• Analyzing Leakage of Personally Identifiable Information in Language Models
• SoK: Let the Privacy Games Begin! A Unified Treatment of Data Inference Privacy in Machine Learning
• Attacks on Deidentification's Defenses
• Defending against Reconstruction Attacks with Rényi Differential Privacy
• AIM: An Adaptive and Iterative Mechanism for Differentially Private Synthetic Data
• Reconstructing Training Data with Informed Adversaries
• Large Language Models Can Be Strong Differentially Private Learners
• Winning the NIST Contest: A scalable and general approach to differentially private synthetic data
• Differentially Private Query Release Through Adaptive Projection
• Extracting Training Data from Large Language Models
• Investigating Membership Inference Attacks under Data Dependencies
• Auditing Differentially Private Machine Learning: How Private is Private SGD?
• Linear Program Reconstruction in Practice
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Membership Inference Attacks against Machine Learning Models
• The Composition Theorem for Differential Privacy

Cited by

• Observational Auditing of Label Privacy
• Unifying Re-Identification, Attribute Inference, and Data Reconstruction Risks in Differential Privacy

Related

• ATTAXONOMY: Unpacking Differential Privacy Guarantees Against Practical Adversaries
• Unifying Re-Identification, Attribute Inference, and Data Reconstruction Risks in Differential Privacy
• From Mean to Extreme: Formal Differential Privacy Bounds on the Success of Real-World Data Reconstruction Attacks
• Why Does Differential Privacy with Large Epsilon Defend Against Practical Membership Inference Attacks?
• Reconstruction Attacks on Aggressive Relaxations of Differential Privacy