Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs

2024/06/14 by Abhimanyu Hans, Yuxin Wen, Hans, Abhimanyu +19 · 8 citations
Computer Science · Social Sciences · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling #Wikis in Education and Collaboration

paper · pdf · doi:10.48550/arxiv.2406.10209

openalex publication_date 2024/06/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/31

Abstract

Large language models can memorize and repeat their training data, causing privacy and copyright risks. To mitigate memorization, we introduce a subtle modification to the next-token training objective that we call the goldfish loss. During training, randomly sampled subsets of tokens are excluded from the loss computation. These dropped tokens are not memorized by the model, which prevents verbatim reproduction of a complete chain of tokens from the training set. We run extensive experiments training billion-scale Llama-2 models, both pre-trained and trained from scratch, and demonstrate significant reductions in extractable memorization with little to no impact on downstream benchmarks.

Cited by

• In-Context Learning Without Copying
• RECAP: Reproducing Copyrighted Data from LLMs Training with an Agentic Pipeline
• Early Detection and Reduction of Memorisation for Domain Adaptation and Instruction Tuning
• MultiMat: Multimodal Program Synthesis for Procedural Materials using Large Multimodal Models
• Crossing the Margin Cliff: Toward Relearn-Robust LLM Unlearning via Margin Calibration
• Incident Analysis for AI Agents
• Unveiling Over-Memorization in Finetuning LLMs for Reasoning Tasks
• Win-k: Improved Membership Inference Attacks on Small Language Models

Related

• A Lightweight Method to Disrupt Memorized Sequences in LLM
• To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Models
• Memorization or Interpolation ? Detecting LLM Memorization through Input Perturbation Analysis
• The Landscape of Memorization in LLMs: Mechanisms, Measurement, and Mitigation
• Quantifying Memorization Across Neural Language Models