2025/12/18 by Zhengyu Lu, Zhexi Lu, Lu, Zhexi +11
Computer Science · #Privacy-Preserving Technologies in Data #Topic Modeling #Adversarial Robustness in Machine Learning
paper · pdf · doi:10.14722/ndss.2026.240892
membership inference [5], [6].Among these stages, finetuning is particularly susceptible to privacy leaks, due to the typically limited size and sensitive nature of the fine-tuning datasets [7].Membership inference attacks (MIAs) aim to determine whether a particular data sample was part of a model's training dataset, thereby potentially revealing sensitive or personally identifiable information about individuals [8].MIAs have been extensively studied across a range of machine learning domains to identify and characterize privacy risks.These include generative adversarial networks (GANs) [9], explainable machine learning models [10], and diffusion models [11].In addition to vulnerability assessment, MIAs have also been employed to evaluate the efficacy of privacy-preserving mechanisms [12], [13], benchmark machine unlearning methods [14], and enable privacy auditing in deployed systems [15]-[17].Recently, a growing body of work [18], [19] has adapted MIAs to assess the privacy risks of LLMs.These efforts build on classical MIA techniques but tailor them to the unique properties of LLMs.Broadly, existing MIA methods for LLMs can be categorized into two classes: reference-based attacks, which rely on an auxiliary/reference dataset (typically drawn from a distribution similar to the model's training data) to train one or more reference models, and referencefree attacks, which avoid this requirement.Reference-based attacks, such as those by Mireshghallah et al. [15], [18], extend the Likelihood Ratio Attack (LiRA) framework [20] to LLMs and masked language models.These attacks require training an ensemble of shadow models and comparing the target model's negative log-likelihood (NLL) on a given sample against the distribution of NLLs from these shadow models to determine the membership.However, such approaches assume access to auxiliary data from the same distribution as the target model's training set, an assumption that rarely holds in real-world scenarios, especially when fine-tuning involves private or proprietary datasets.In contrast, reference-free attacks such as Min-K% [5] and Min-K%++ [21] detect memorized samples by identifying low-rank (outlier) tokens in the model's output, which are indicative of overfitting.ReCaLL [22] demonstrates that adding a context prefix to the input can differentially affect the model's predictions for memorized versus non-memorized Abstract-Membership inference attacks (MIAs) pose a critical privacy threat to fine-tuned large language models (LLMs), especially when models are adapted to domain-specific tasks using sensitive data.While prior black-box MIA techniques rely on confidence scores or token likelihoods, these signals are often entangled with a sample's intrinsic properties-such as content difficulty or rarity-leading to poor generalization and low signal-to-noise ratios.In this paper, we propose ICP-MIA, a novel MIA framework grounded in the theory of training dynamics, particularly the phenomenon of diminishing returns during optimization.We introduce the Optimization Gap as a fundamental signal of membership: at convergence, member samples exhibit minimal remaining loss-reduction potential, while non-members retain significant potential for further optimization.To estimate this gap in a black-box setting, we propose In-Context Probing (ICP)-a training-free method that simulates fine-tuning-like behavior via strategically constructed input contexts.We propose two probing strategies: reference-data-based (using semantically similar public samples) and self-perturbation (via masking or generation).Experiments on three tasks and multiple LLMs show that ICP-MIA significantly outperforms prior black-box MIAs, particularly at low false positive rates.We further analyze how reference data alignment, model type, PEFT configurations, and training schedules affect attack effectiveness.Our findings establish ICP-MIA as a practical and theoretically grounded framework for auditing privacy risks in deployed LLMs.