Offset Unlearning for Large Language Models

2024/04/17 by James Y. Huang, Wenxuan Zhou, Huang, James Y. +11 · 10 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2404.11045

openalex publication_date 2024/04/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/03

Abstract

Despite the strong capabilities of Large Language Models (LLMs) to acquire knowledge from their training corpora, the memorization of sensitive information in the corpora such as copyrighted, biased, and private content has led to ethical and legal concerns. In response to these challenges, unlearning has emerged as a potential remedy for LLMs affected by problematic training data. However, previous unlearning techniques are either not applicable to black-box LLMs due to required access to model internal weights, or violate data protection principles by retaining sensitive data for inference-time correction. We propose δ-Unlearning, an offset unlearning framework for black-box LLMs. Instead of tuning the black-box LLM itself, δ-Unlearning learns the logit offset needed for unlearning by contrasting the logits from a pair of smaller models. Experiments demonstrate that δ- Unlearning can effectively unlearn target data while maintaining similar or even stronger performance on general out-of-forget-scope tasks. δ-Unlearning also effectively incorporates different unlearning algorithms, making our approach a versatile solution to adapting various existing unlearning algorithms to black-box LLMs.

Cited by

• Recover-to-Forget: Gradient Reconstruction from LoRA for Efficient LLM Unlearning
• Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shifting
• Logit Arithmetic Elicits Long Reasoning Capabilities Without Training
• OFMU: Optimization-Driven Framework for Machine Unlearning
• Logit Arithmetic Elicits Long Reasoning Capabilities Without Training
• SoK: Machine Unlearning for Large Language Models
• Align-then-Unlearn: Embedding Alignment for LLM Unlearning
• Lifting Data-Tracing Machine Unlearning to Knowledge-Tracing for Foundation Models
• Exploring Criteria of Loss Reweighting to Enhance LLM Unlearning
• Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks

Related

• Unlearn What You Want to Forget: Efficient Unlearning for LLMs
• A Closer Look at Machine Unlearning for Large Language Models
• DP2Unlearning: An Efficient and Guaranteed Unlearning Framework for LLMs
• In-Context Unlearning: Language Models as Few Shot Unlearners
• Large Language Model Unlearning