Enhancing One-run Privacy Auditing with Quantile Regression-Based Membership Inference

2025/06/18 by Terrance Liu, Liu, Terrance, Matteo Boglioni +9
Computer Science · Decision Sciences · Social Sciences · #Cryptography and Security (cs.CR) #Data Quality and Management #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy, Security, and Data Protection #Privacy-Preserving Technologies in Data

paper · pdf · doi:10.48550/arxiv.2506.15349

openalex publication_date 2025/06/18 · openalex created_date 2025/10/19 · openalex updated_date 2026/07/28

Abstract

Differential privacy (DP) auditing aims to provide empirical lower bounds on the privacy guarantees of DP mechanisms like DP-SGD. While some existing techniques require many training runs that are prohibitively costly, recent work introduces one-run auditing approaches that effectively audit DP-SGD in white-box settings while still being computationally efficient. However, in the more practical black-box setting where gradients cannot be manipulated during training and only the last model iterate is observed, prior work shows that there is still a large gap between the empirical lower bounds and theoretical upper bounds. Consequently, in this work, we study how incorporating approaches for stronger membership inference attacks (MIA) can improve one-run auditing in the black-box setting. Evaluating on image classification models trained on CIFAR-10 with DP-SGD, we demonstrate that our proposed approach, which utilizes quantile regression for MIA, achieves tighter bounds while crucially maintaining the computational efficiency of one-run methods.

Citations

• Auditing f-Differential Privacy in One Run
• The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD
• Tighter Privacy Auditing of DP-SGD in the Hidden State Threat Model
• Nearly Tight Black-Box Auditing of Differentially Private Machine Learning
• Auditing Private Prediction
• Membership Inference Attacks on Diffusion Models via Quantile Regression
• Scalable Membership Inference Attacks via Quantile Regression
• Unleashing the Power of Randomization in Auditing Differentially Private ML
• Privacy Auditing with One (1) Training Run
• Unlocking High-Accuracy Differentially Private Image Classification through Scale
• Debugging Differential Privacy: A Case Study for Privacy Auditing
• A ConvNet for the 2020s
• A ConvNet for the 2020s
• Membership Inference Attacks From First Principles
• Auditing Differentially Private Machine Learning: How Private is Private SGD?
• Membership Inference Attacks against Machine Learning Models
• Wide Residual Networks
• Differentially Private Empirical Risk Minimization

Related

• Auditing Differential Privacy Guarantees Using Density Estimation
• Optimizing Canaries for Privacy Auditing with Metagradient Descent
• Bounding Membership Inference
• Nearly Tight Black-Box Auditing of Differentially Private Machine Learning
• How Well Can Differential Privacy Be Audited in One Run?