Data Compressibility Quantifies LLM Memorization

2025/07/08 by Zhe Yang, Huang, Yizhan, Yang, Zhe +8 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning in Healthcare #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2507.06056

openalex publication_date 2025/07/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large Language Models (LLMs) are known to memorize portions of their training data, sometimes even reproduce content verbatim when prompted appropriately. Despite substantial interest, existing LLM memorization research has offered limited insight into how training data influences memorization and largely lacks quantitative characterization. In this work, we build upon the line of research that seeks to quantify memorization through data compressibility. We analyze why prior attempts fail to yield a reliable quantitative measure and show that a surprisingly simple shift from instance-level to set-level metrics uncovers a robust phenomenon, which we term the Entropy--Memorization (EM) Linearity. This law states that a set-level data entropy estimator exhibits a linear correlation with memorization scores.

Citations

• How much do language models memorize?
• Exploring the limits of strong membership inference attacks on large language models
• Extracting memorized pieces of (copyrighted) books from open-weight language models
• SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
• 2 OLMo 2 Furious
• Measuring Non-Adversarial Reproduction of Training Data in Large Language Models
• Demystifying Verbatim Memorization in Large Language Models
• Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data
• AI Benchmark Half-Life in Recursive Corpora: A Theory of Validity Decay under Semantic Leakage and Regeneration
• Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon
• Noisy Neighbors: Efficient membership inference attacks against LLMs
• LLM Dataset Inference: Did you train on my dataset?
• Unveiling Differences in Generative Models: A Scalable Differential Clustering Approach
• Rethinking LLM Memorization through the Lens of Adversarial Compression
• Multicalibration for Confidence Scoring in LLMs
• Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models
• Do Membership Inference Attacks Work on Large Language Models?
• OLMo: Accelerating the Science of Language Models
• Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
• Quantifying and Analyzing Entity-level Memorization in Large Language Models
• ProPILE: Probing Privacy Leakage in Large Language Models
• Membership Inference Attacks against Language Models via Neighbourhood Comparison
• Emergent and Predictable Memorization in Large Language Models
• A Note on Normalized Emergence Timing (in Pythia Language Model Evaluations)
• Targeted Attack on GPT-Neo for the SATML Language Model Data Extraction Challenge
• Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy
• Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models
• Training language models to follow instructions with human feedback
• Quantifying Memorization Across Neural Language Models
• Deduplicating Training Data Mitigates Privacy Risks in Language Models
• Counterfactual Memorization in Neural Language Models
• Membership Inference Attacks From First Principles
• How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN
• Dataset Inference: Ownership Resolution in Machine Learning
• Membership Inference Attack Susceptibility of Clinical Language Models
• The Pile: An 800GB Dataset of Diverse Text for Language Modeling
• Extracting Training Data from Large Language Models
• What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation
• Scaling Laws for Neural Language Models
• Does Learning Require Memorization? A Short Tale about a Long Tail
• The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
• Understanding Black-box Predictions via Influence Functions
• Understanding deep learning requires rethinking generalization
• Least squares quantization in PCM

Cited by

• Extracting alignment data in open models

Related

• Entropy Law: The Story Behind Data Compression and LLM Performance
• Understanding Memorisation in LLMs: Dynamics, Influencing Factors, and Implications
• A Multi-Perspective Analysis of Memorization in Large Language Models
• Get Confused Cautiously: Textual Sequence Memorization Erasure with Selective Entropy Maximization
• Quantifying Memorization Across Neural Language Models