Pruning as a Defense: Reducing Memorization in Large Language Models

2025/02/18 by Mansi Gupta, Nikhar Waghela, Gupta, Mansi +6 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2502.15796

openalex publication_date 2025/02/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large language models have been shown to memorize significant portions of their training data, which they can reproduce when appropriately prompted. This work investigates the impact of simple pruning techniques on this behavior. Our findings reveal that pruning effectively reduces the extent of memorization in LLMs, demonstrating its potential as a foundational approach for mitigating membership inference attacks.

Cited by

• Bits and Memories: Measuring Verbatim Extraction Across LLM Quantization

Related

• Impact of Fine-Tuning Methods on Memorization in Large Language Models
• The Landscape of Memorization in LLMs: Mechanisms, Measurement, and Mitigation
• Mitigating Memorization In Language Models
• Quantifying Memorization Across Neural Language Models
• Towards Robust Pruning: An Adaptive Knowledge-Retention Pruning Strategy for Language Models