Machine Unlearning
2019/12/09 by Lucas Bourtoule, Varun Chandrasekaran, Bourtoule, Lucas +13 · 3 voices · 280 citations
Computer Science · Decision Sciences · #Age of Information Optimization #Artificial intelligence #Artificial neural network #Computer science #Data Quality and Management #Machine learning #Overhead (engineering) #Point (geometry) #Privacy-Preserving Technologies in Data #Process (computing) #Retraining #Stochastic gradient descent #cs.AI #cs.CR #cs.LG
paper · pdf · doi:10.48550/arxiv.1912.03817
published in arXiv (Cornell University) (Cornell University) · Published in IEEE S&P 2021
openalex publication_date 2019/12/09 · arxiv created 2020/12/15 · arxiv updated 2020/12/16 · openalex created_date 2022/10/01 · openalex updated_date 2026/07/28
Abstract
Once users have shared their data online, it is generally difficult for them to revoke access and ask for the data to be deleted. Machine learning (ML) exacerbates this problem because any model trained with said data may have memorized it, putting users at risk of a successful privacy attack exposing their information. Yet, having models unlearn is notoriously difficult. We introduce SISA training, a framework that expedites the unlearning process by strategically limiting the influence of a data point in the training procedure. While our framework is applicable to any learning algorithm, it is designed to achieve the largest improvements for stateful algorithms like stochastic gradient descent for deep neural networks. SISA training reduces the computational overhead associated with unlearning, even in the worst-case setting where unlearning requests are made uniformly across the training set. In some cases, the service provider may have a prior on the distribution of unlearning requests that will be issued by users. We may take this prior into account to partition and order data accordingly, and further decrease overhead from unlearning. Our evaluation spans several datasets from different domains, with corresponding motivations for unlearning. Under no distributional assumptions, for simple learning tasks, we observe that SISA training improves time to unlearn points from the Purchase dataset by 4.63x, and 2.45x for the SVHN dataset, over retraining from scratch. SISA training also provides a speed-up of 1.36x in retraining for complex learning tasks such as ImageNet classification; aided by transfer learning, this results in a small degradation in accuracy. Our work contributes to practical data governance in machine unlearning.
Cited by
- DCS: A Unified Conditional Sensitivity Framework for Cross-Modal Copyright Infringement Detection
- Unlearning Under Imbalance: Benchmarking Fairness in Multimodal LLM Unlearning
- Unlearning as Distribution Restoration: A Controlled Counterfactual Study, a Validated Selective Screen, and the Limits of Oracle-Free Certification
- Falsifiable Release Gates for Self-Improving Systems: Standing Invariants at Scale
- Stochastic Meta-Unlearning: Bridging Language Backbone and Multimodal Unlearning
- Understanding How University Guidelines Address Privacy and Security Issues of Generative AI in Academic Settings
- Quantifying Training Membership Information in the Hyperspherical Embedding Geometry of Face Recognition Models
- When Machine Unlearning Meets Retrieval-Augmented Generation (RAG): Keep Secret or Forget Knowledge?
- Not All Tokens Are Meant to Be Forgotten
- Not All Data Are Unlearned Equally
- Privacy Ripple Effects from Adding or Removing Personal Information in Language Model Training
- Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps
- Certifying the Right to Be Forgotten: Primal-Dual Optimization for Sample and Label Unlearning in Vertical Federated Learning
- Understanding Machine Unlearning Through the Lens of Mode Connectivity
- DECAF: De-Clustering for Adaptive Representational Unlearning
- Doc-to-LoRA: Learning to Instantly Internalize Contexts
- Obliviate: Efficient Unlearning in Recommender Systems
- A Mechanistic Perspective and Circuit-Guided Difficulty Metric for Unlearning
- Investigating Model Editing for Unlearning in Large Language Models
- Machine Unlearning in the Era of Quantum Machine Learning: An Empirical Study
- Towards Benchmarking Privacy Vulnerabilities in Selective Forgetting with Large Language Models
- Dual-View Inference Attack: Machine Unlearning Amplifies Privacy Exposure
- Metanetworks as Regulatory Operators: Learning to Edit for Requirement Compliance
- FAME: Fictional Actors for Multilingual Erasure
- Erasing CLIP Memories: Non-Destructive, Data-Free Zero-Shot class Unlearning in CLIP Models
- Selective, Controlled and Domain-Agnostic Unlearning in Pretrained CLIP: A Training- and Data-Free Approach
- Face Identity Unlearning for Retrieval via Embedding Dispersion
- Forgetful but Faithful: A Cognitive Memory Architecture and Benchmark for Privacy-Aware Generative Agents
- Sparse Concept Anchoring for Interpretable and Controllable Neural Representations
- Natural Geometry of Robust Data Attribution: From Convex Models to Deep Networks
- ZK-APEX: Zero-Knowledge Approximate Personalized Unlearning with Executable Proofs
- Forget and Explain: Transparent Verification of GNN Unlearning
- LUNE: Efficient LLM Unlearning via LoRA Fine-Tuning with Negative Examples
- Recover-to-Forget: Gradient Reconstruction from LoRA for Efficient LLM Unlearning
- Memory Power Asymmetry in Human-AI Relationships: Preserving Mutual Forgetting in the Digital Age
- SUGAR: A Sweeter Spot for Generative Unlearning of Many Identities
- RapidUn: Influence-Driven Parameter Reweighting for Efficient Large Language Model Unlearning
- Efficient Public Verification of Private ML via Regularization
- Grokked Models are Better Unlearners
- Adaptive-lambda Subtracted Importance Sampled Scores in Machine Unlearning for DDPMs and VAEs
- Teleportation-Based Defenses for Privacy in Approximate Machine Unlearning
- FedSGT: Exact Federated Unlearning via Sequential Group-based Training
- Illuminating the Black Box: Real-Time Monitoring of Backdoor Unlearning in CNNs via Explainable AI
- ModHiFi: Identifying High Fidelity predictive components for Model Modification
- POUR: A Provably Optimal Method for Unlearning Representations via Neural Collapse
- SineProject: Machine Unlearning for Stable Vision Language Alignment
- Curvature-Aware Safety Restoration In LLMs Fine-Tuning
- SG-OIF: A Stability-Guided Online Influence Framework for Reliable Vision Data
- Geometric-disentangelment Unlearning
- Membership Inference Attacks Beyond Overfitting
- Erase to Retain: Low Rank Adaptation Guided Selective Unlearning in Medical Segmentation Networks
- Beyond Tokens in Language Models: Interpreting Activations through Text Genre Chunks
- Selective Forgetting in Option Calibration: An Operator-Theoretic Gauss-Newton Framework
- Coffee: Controllable Diffusion Fine-tuning
- Forgetting-MarI: LLM Unlearning via Marginal Information Regularization
- Learning to Fast Unrank in Collaborative Filtering Recommendation
- Beyond Uniform Deletion: A Data Value-Weighted Framework for Certified Machine Unlearning
- FiCABU: A Fisher-Based, Context-Adaptive Machine Unlearning Processor for Edge AI
- The Realignment Problem: When Right becomes Wrong in LLMs
- Improving Unlearning with Model Updates Probably Aligned with Gradients
- Quantum Machine Unlearning: Foundations, Mechanisms, and Taxonomy
- MPRU: Modular Projection-Redistribution Unlearning as Output Filter for Classification Pipelines
- Signed Graph Unlearning
- Multistakeholder Impacts of Profile Portability in a Recommender Ecosystem
- A Survey on Unlearning in Large Language Models
- On the Impossibility of Retrain Equivalence in Machine Unlearning
- Label Smoothing Improves Gradient Ascent in LLM Unlearning
- Efficient Utility-Preserving Machine Unlearning with Implicit Gradient Surgery
- Leverage Unlearning to Sanitize LLMs
- LEGO: A Lightweight and Efficient Multiple-Attribute Unlearning Framework for Recommender Systems
- LLM Unlearning with LLM Beliefs
- Not Every Time and Frequency Need to Be Forgotten in Diffusion Unlearning
- Backdoor Unlearning by Linear Task Decomposition
- Gaussian Certified Unlearning in High Dimensions: A Hypothesis Testing Approach
- Evaluating the Quality of Randomness and Entropy in Tasks Supported by Large Language Models
- Approximate Domain Unlearning for Vision-Language Models
- Federated Unlearning in the Wild: Rethinking Fairness and Data Discrepancy
- Cross-Modal Attention Guided Unlearning in Vision-Language Models
- Distribution Preference Optimization: A Fine-grained Perspective for LLM Unlearning
- Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models
- Machine Unlearning in Speech Emotion Recognition via Forget Set Alone
- Unlearning in Diffusion models under Data Constraints: A Variational Inference Approach
- Direct Token Optimization: A Self-contained Approach to Large Language Model Unlearning
- Rotation Control Unlearning: Quantifying and Controlling Continuous Unlearning for LLM with The Cognitive Rotation Space
- SMS: Self-supervised Model Seeding for Verification of Machine Unlearning
- Ascent Fails to Forget
- Understanding the Dilemma of Unlearning for Large Language Models
- Preserving Cross-Modal Stability for Visual Unlearning in Multimodal Scenarios
- Copyright Infringement Detection in Text-to-Image Diffusion Models via Differential Privacy
- OFMU: Optimization-Driven Framework for Machine Unlearning
- Can Prompts Rewind Time for LLMs? Evaluating the Effectiveness of Prompted Knowledge Cutoffs
- A Unified Framework for Diffusion Model Unlearning with f-Divergence
- Beyond Sharp Minima: Robust LLM Unlearning via Feedback-Guided Multi-Point Optimization
- CURE: Centroid-guided Unsupervised Representation Erasure for Facial Recognition Systems
- Benign on Label, Malicious by Design: Clean-Label Dormant-to-Activated Backdoor via Machine Unlearning with Removable Camouflage
- Subtract or Replay? Exact Deletion from Language-Model Memory
- Beyond Binary Rewards: A Comparative Study of Reward Design for Reinforcement Unlearning
- Memory in Large Language Models: Mechanisms, Evaluation and Evolution
- TraceHiding: Scalable Machine Unlearning for Mobility Data
- Scrub It Out! Erasing Sensitive Memorization in Code Language Models via Machine Unlearning
- ReTrack: Data Unlearning in Diffusion Models through Redirecting the Denoising Trajectory
- Forget What's Sensitive, Remember What Matters: Token-Level Differential Privacy in Memory Sculpting for Continual Learning
- Module-Aware Parameter-Efficient Machine Unlearning on Transformers
- Membership Inference Attacks on Recommender System: A Survey
- Machine Unlearning for Responsible and Adaptive AI in Education
- Customized Retrieval-Augmented Generation with LLM for Debiasing Recommendation Unlearning
- Apollo: A Posteriori Label-Only Membership Inference Attack Towards Machine Unlearning
- Bias-Aware Machine Unlearning: Towards Fairer Vision Models via Controllable Forgetting
- zkUnlearner: A Zero-Knowledge Framework for Verifiable Unlearning with Multi-Granularity and Forgery-Resistance
- MRD-LiNet: A Novel Lightweight Hybrid CNN with Gradient-Guided Unlearning for Improved Drought Stress Identification
- From Membership-Privacy Leakage to Quantum Machine Unlearning
- The Measure of Deception: An Analysis of Data Forging in Machine Unlearning
- Pre-Forgettable Models: Prompt Learning as a Native Mechanism for Unlearning
- Graph Unlearning: Efficient Node Removal in Graph Neural Networks
- Evaluating the Defense Potential of Machine Unlearning against Membership Inference Attacks
- AMCR: A Framework for Assessing and Mitigating Copyright Risks in Generative Models
- FUTURE: Flexible Unlearning for Tree Ensemble
- Towards Mitigating Excessive Forgetting in LLM Unlearning via Entanglement-Guidance with Proxy Constraint
- MobText-SISA: Efficient Machine Unlearning for Mobility Logs with Spatio-Temporal and Natural-Language Data
- Tackling Federated Unlearning as a Parameter Estimation Problem
- Auditing Approximate Machine Unlearning for Differentially Private Models
- Data Augmentation Improves Machine Unlearning
- Curriculum Approximate Unlearning for Session-based Recommendation
- BadFU: Backdoor Federated Learning through Adversarial Machine Unlearning
- Reliable Unlearning Harmful Information in LLMs with Metamorphosis Representation Projection
- Towards Source-Free Machine Unlearning
- Side Effects of Erasing Concepts from Diffusion Models
- FedUP: Efficient Pruning-based Federated Unlearning for Model Poisoning Attacks
- Involuntary Jailbreak: On Self-Prompting Attacks
- Unlearning at Scale: Implementing the Right to be Forgotten in Large Language Models
- Demystifying Foreground-Background Memorization in Diffusion Models
- Slow Tuning and Low-Entropy Masking for Safe Chain-of-Thought Distillation
- Invisible Watermarks, Visible Gains: Steering Machine Unlearning with Bi-Level Watermarking Design
- EFU: Enforcing Federated Unlearning via Functional Encryption
- Revisiting Data Attribution for Influence Functions
- Towards Unveiling Predictive Uncertainty Vulnerabilities in the Context of the Right to Be Forgotten
- Membership Inference Attacks with False Discovery Rate Control
- Membership Inference Attack with Partial Features
- Integrated Influence: Data Attribution with Baseline
- WSS-CL: Weight Saliency Soft-Guided Contrastive Learning for Efficient Machine Unlearning Image Classification
- Conformal Unlearning: A New Paradigm for Unlearning in Conformal Predictors
- Superior resilience to poisoning and amenability to unlearning in quantum machine learning
- Graph Unlearning via Embedding Reconstruction -- A Range-Null Space Decomposition Approach
- IMU: Influence-guided Machine Unlearning
- Towards Evaluation for Real-World LLM Unlearning
- Challenges of Trustworthy Federated Learning: What's Done, Current Trends and Remaining Work
- Efficient Machine Unlearning via Influence Approximation
- LoReUn: Data Itself Implicitly Provides Cues to Improve Machine Unlearning
- Zero-Shot Machine Unlearning with Proxy Adversarial Data Generation
- Reminiscence Attack on Residuals: Exploiting Approximate Machine Unlearning for Privacy
- Unlearning of Knowledge Graph Embedding via Preference Optimization
- Machine Unlearning for Streaming Forgetting
- A Survey on Generative Model Unlearning: Fundamentals, Taxonomy, Evaluation, and Future Direction
- The Right to be Forgotten in Pruning: Unveil Machine Unlearning on Sparse Models
- Distributional Machine Unlearning via Selective Data Removal
- Machine Unlearning of Traffic State Estimation and Prediction
- Continual Speaker Identity Unlearning with Minimal Interference
- Shaping capabilities with token-level data filtering
- What Should LLMs Forget? Quantifying Personal Data in LLMs for Right-to-Be-Forgotten Requests
- Enhancing Safe and Controllable Protein Generation via Knowledge Preference Optimization
- How to Protect Models against Adversarial Unlearning?
- SoK: Machine Unlearning for Large Language Models
- Memorization Sinks: Isolating Memorization during LLM Training
- Leveraging Distribution Matching to Make Approximate Machine Unlearning Faster
- MMFGU: Multimodal Federated Graph Unlearning
- Meta-Learning Transformers to Improve In-Context Generalization
- Efficient Unlearning with Privacy Guarantees
- Model Collapse Is Not a Bug but a Feature in Machine Unlearning for LLMs
- LoRAShield: Data-Free Editing Alignment for Secure Personalized LoRA Sharing
- SecureT2I: No More Unauthorized Manipulation on AI Generated Images from Prompts
- Unlearning the Noisy Correspondence Makes CLIP More Robust
- NOVO: Unlearning-Compliant Vision Transformers
- Rethinking Data Protection in the (Generative) Artificial Intelligence Era
- Meaningful Data Erasure in the Presence of Dependencies
- Forget-MI: Machine Unlearning for Forgetting Multimodal Information in Healthcare Settings
- Revisiting the Past: Data Unlearning with Model State History
- On the Necessity of Output Distribution Reweighting for Effective Class Unlearning
- Orthogonal Soft Pruning for Efficient Class Unlearning
- Recalling The Forgotten Class Memberships: Unlearned Models Can Be Noisy Labelers to Leak Privacy
- Large Language Model Unlearning for Source Code
- Towards Reliable Forgetting: A Survey on Machine Unlearning Verification
- PDLRecover: Privacy-preserving Decentralized Model Recovery with Machine Unlearning
- Learning-Time Encoding Shapes Unlearning in LLMs
- Train Once, Forget Precisely: Anchored Optimization for Efficient Post-Hoc Unlearning
- Unlearning Isn't Invisible: Detecting Unlearning Traces in LLMs from Model Outputs
- Membership Inference Attacks as Privacy Tools: Reliability, Disparity and Ensemble
- Unlearning-Enhanced Website Fingerprinting Attack: Against Backdoor Poisoning in Anonymous Networks
- Rectifying Privacy and Efficacy Measurements in Machine Unlearning: A New Inference Attack Perspective
- Certified Unlearning for Neural Networks
- Machine Unlearning for Robust DNNs: Attribution-Guided Partitioning and Neuron Pruning in Noisy Environments
- UCD: Unlearning in LLMs via Contrastive Decoding
- Lifting Data-Tracing Machine Unlearning to Knowledge-Tracing for Foundation Models
- System-Aware Unlearning Algorithms: Use Lesser, Forget Faster
- Distillation Robustifies Unlearning
- A Certified Unlearning Approach without Access to Source Data
- Constrained Entropic Unlearning: A Primal-Dual Framework for Large Language Models
- SECNEURON: Reliable and Flexible Abuse Control in Local LLMs via Hybrid Neuron Encryption
- Quantifying Cross-Modality Memorization in Vision-Language Models
- FictionalQA: A Dataset for Studying Memorization and Knowledge Acquisition
- OBLIVIATE: Robust and Practical Machine Unlearning for Large Language Models
- Lacuna Inc. at SemEval-2025 Task 4: LoRA-Enhanced Influence-Based Unlearning for LLMs
- Matter-of-Fact: A Benchmark for Verifying the Feasibility of Literature-Supported Claims in Materials Science
- Memory-Efficient Distributed Unlearning
- Rethinking Post-Unlearning Behavior of Large Vision-Language Models
- Targeted Forgetting of Image Subgroups in CLIP Models
- Rethinking Machine Unlearning in Image Generation Models
- Unlearning's Blind Spots: Over-Unlearning and Prototypical Relearning Attack
- Unlearning Inversion Attacks for Graph Neural Networks
- Speech Unlearning
- Existing Large Language Model Unlearning Evaluations Are Inconclusive
- Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy
- Does Machine Unlearning Truly Remove Knowledge?
- Pre-training for Recommendation Unlearning
- BLUR: A Benchmark for LLM Unlearning Robust to Forget-Retain Overlap
- Machine Unlearning under Overparameterization
- From Dormant to Deleted: Tamper-Resistant Unlearning Through Weight-Space Regularization
- Graceful Forgetting in Generative Language Models
- Editing as Unlearning: Are Knowledge Editing Methods Strong Baselines for Large Language Model Unlearning?
- Safety Alignment via Constrained Knowledge Unlearning
- Leveraging Per-Instance Privacy for Machine Unlearning
- Exploring and Bridging Knowledge Holes in Unlearned Multimodal Large Language Models
- A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advances
- T2VUnlearning: A Concept Erasing Method for Text-to-Video Diffusion Models
- CTRAP: Embedding Collapse Trap to Safeguard Large Language Models from Harmful Fine-Tuning
- Unlearning Isn't Deletion: Investigating Reversibility of Machine Unlearning in LLMs
- Losing is for Cherishing: Data Valuation Based on Machine Unlearning and Shapley Value
- SCOPE: Entanglement Frontier Escape for Source-Free Class Unlearning
- On the creation of narrow AI: hierarchy and nonlocality of neural network skills
- UniErase: Towards Balanced and Precise Unlearning in Language Models
- A Unified Gradient-based Framework for Task-agnostic Continual Learning-Unlearning
- Unlearning Algorithmic Biases over Graphs
- SEPS: A Separability Measure for Robust Unlearning in LLMs
- QR-Erase: Efficient Subspace-Based Machine Unlearning with Layer Localization
- Towards Efficient and Exact Forgetting Services in Pre-Trained-Model-based Continual Learning
- Exploring Criteria of Loss Reweighting to Enhance LLM Unlearning
- Ready2Unlearn: A Learning-Time Approach for Preparing Models with Future Unlearning Readiness
- Generated Images Are Easier to Forget: A Machine Unlearning Perspective for Synthetic Image Detection
- AC-LoRA: (Almost) Training-Free Access Control-Aware Multi-Modal LLMs
- Similarity-Aware Machine Unlearning
- Enabling Group Fairness in Graph Unlearning via Bi-level Debiasing
- MUBox: A Critical Evaluation Framework of Deep Machine Unlearning
- Online Learning and Unlearning
- Mirror Mirror on the Wall, Have I Forgotten it All? A New Framework for Evaluating Machine Unlearning
- Certified Data Removal Under High-dimensional Settings
- Efficient Machine Unlearning by Model Splitting and Core Sample Selection
- PRUNE: A Patching Based Repair Framework for Certifiable Unlearning of Neural Networks
- WaterDrum: Watermarking for Data-centric Unlearning Metric
- LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearning
- Class Unlearning via Depth-Aware Removal of Forget-Specific Directions
- Auditing Language Model Unlearning via Information Decomposition
- Multimodal Unlearning Across Vision, Language, Video, and Audio: Survey of Methods, Datasets, and Benchmarks
- Governable Individuals: An Identity Layer for Embodied Agents That Keep Learning
- Memory for Autonomous LLM Agents:Mechanisms, Evaluation, and Emerging Frontiers
- Exact Unlearning in Reinforcement Learning
- Faults and Pitfalls in Implementing the Right to be Forgotten
- Interpretability Can Be Actionable
- Deployment-Time Memorization in Foundation-Model Agents
- Large Language Models Generate Harmful Responses Using a Distinct Mechanism, Shared Across Harm Types
- An Illusion of Unlearning? Assessing Machine Unlearning Through Internal Representations
- Per-parameter Task Arithmetic for Unlearning in Large Language Models
- Representation-Aware Unlearning via Activation Signatures: From Suppression to Entity-Signature Erasure
- Suppression Sticks, Locality Is Fragile: A Closed-Loop Target-and-Control Audit of Task-Vector Negation in VLA Policies
- Forgetful Attention: An Auditable Support-Vector Memory for Selective Retention and Verified Deletion
- Markov Chain Monte Carlo-Based Machine Unlearning: Unlearning What Needs to be Forgotten
- Exact Unlearning of Finetuning Data via Model Merging at Scale
- DualOptim: Enhancing Efficacy and Stability in Machine Unlearning with Dual Optimizers
- Backdoor Defense in Diffusion Models via Spatial Attention Unlearning
- Verifying Robust Unlearning: Probing Residual Knowledge in Unlearned Models
- DP2Unlearning: An Efficient and Guaranteed Unlearning Framework for LLMs
- A mean teacher algorithm for unlearning of language models
- GROM: Gradient-Free Rapid One-Shot Machine Unlearning
- TRU: Targeted Reverse Update for Efficient Multimodal Recommendation Unlearning
- LLM Unlearning Reveals a Stronger-Than-Expected Coreset Effect in Current Benchmarks
- Sculpting Memory: Multi-Concept Forgetting in Diffusion Models via Dynamic Mask and Concept-Aware Optimization
- Preserving Privacy Without Compromising Accuracy: Machine Unlearning for Handwritten Text Recognition
- SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs
- A Neuro-inspired Interpretation of Unlearning in Large Language Models through Sample-level Unlearning Difficulty
- Sharpness-Aware Parameter Selection for Machine Unlearning
- Inherent and emergent liability issues in LLM-based agentic systems: a principal-agent perspective
- Machine unlearning [wikipedia]
Discussions
Related