Certified Data Removal from Machine Learning Models

2019/11/08 by Chuan Guo, Guo, Chuan, Tom Goldstein +5 · 70 citations
Computer Science · #Privacy-Preserving Technologies in Data #Adversarial Robustness in Machine Learning #Cryptography and Data Security

paper · pdf · doi:10.48550/arxiv.1911.03030

Abstract

Good data stewardship requires removal of data at the request of the data's owner. This raises the question if and how a trained machine-learning model, which implicitly stores information about its training data, should be affected by such a removal request. Is it possible to "remove" data from a machine-learning model? We study this problem by defining certified removal: a very strong theoretical guarantee that a model from which data is removed cannot be distinguished from a model that never observed the data to begin with. We develop a certified-removal mechanism for linear classifiers and empirically study learning settings in which this mechanism is practical.

Citations

• On the limited memory BFGS method for large scale optimization
• YFCC100M

Cited by

• DECAF: De-Clustering for Adaptive Representational Unlearning
• Towards Benchmarking Privacy Vulnerabilities in Selective Forgetting with Large Language Models
• Face Identity Unlearning for Retrieval via Embedding Dispersion
• Forgetful but Faithful: A Cognitive Memory Architecture and Benchmark for Privacy-Aware Generative Agents
• Fully Decentralized Certified Unlearning
• Recover-to-Forget: Gradient Reconstruction from LoRA for Efficient LLM Unlearning
• Efficient Public Verification of Private ML via Regularization
• Stress-Testing Causal Claims via Cardinality Repairs
• Adaptive-lambda Subtracted Importance Sampled Scores in Machine Unlearning for DDPMs and VAEs
• Teleportation-Based Defenses for Privacy in Approximate Machine Unlearning
• Forgetting by Pruning: Data Deletion in Join Cardinality Estimation
• Subtract the Corruption: Training-Data-Free Corrective Machine Unlearning using Task Arithmetic
• POUR: A Provably Optimal Method for Unlearning Representations via Neural Collapse
• Erase to Retain: Low Rank Adaptation Guided Selective Unlearning in Medical Segmentation Networks
• Descend or Rewind? Stochastic Gradient Descent Unlearning
• Selective Forgetting in Option Calibration: An Operator-Theoretic Gauss-Newton Framework
• Synthetic Forgetting without Access: A Few-shot Zero-glance Framework for Machine Unlearning
• Near-Optimal Procedures for Model Discrimination with Non-Disclosure\n Properties
• Forgetting-MarI: LLM Unlearning via Marginal Information Regularization
• Fast Data Attribution for Text-to-Image Models
• Learning to Fast Unrank in Collaborative Filtering Recommendation
• Beyond Uniform Deletion: A Data Value-Weighted Framework for Certified Machine Unlearning
• PrivacyCD: Hierarchical Unlearning for Protecting Student Privacy in Cognitive Diagnosis
• FiCABU: A Fisher-Based, Context-Adaptive Machine Unlearning Processor for Edge AI
• P-MIA: A Profiled-Based Membership Inference Attack on Cognitive Diagnosis Models
• Improving Unlearning with Model Updates Probably Aligned with Gradients
• Quantum Machine Unlearning: Foundations, Mechanisms, and Taxonomy
• MPRU: Modular Projection-Redistribution Unlearning as Output Filter for Classification Pipelines
• Certifiable Machine Unlearning for Linear Models
• On the Impossibility of Retrain Equivalence in Machine Unlearning
• Efficient Utility-Preserving Machine Unlearning with Implicit Gradient Surgery
• LEGO: A Lightweight and Efficient Multiple-Attribute Unlearning Framework for Recommender Systems
• LLM Unlearning with LLM Beliefs
• Gaussian Certified Unlearning in High Dimensions: A Hypothesis Testing Approach
• Approximate Domain Unlearning for Vision-Language Models
• Unlearning in Diffusion models under Data Constraints: A Variational Inference Approach
• KnowledgeSmith: Uncovering Knowledge Updating in LLMs with Model Editing and Unlearning
• Direct Token Optimization: A Self-contained Approach to Large Language Model Unlearning
• SMS: Self-supervised Model Seeding for Verification of Machine Unlearning
• Ascent Fails to Forget
• Understanding the Dilemma of Unlearning for Large Language Models
• Factor Decorrelation Enhanced Data Removal from Deep Predictive Models
• Beyond Sharp Minima: Robust LLM Unlearning via Feedback-Guided Multi-Point Optimization
• Beyond Binary Rewards: A Comparative Study of Reward Design for Reinforcement Unlearning
• Selective Forgetting of Deep Networks at a Finer Level than Samples
• Causal Fuzzing for Verifying Machine Unlearning
• Sparse-Autoencoder-Guided Internal Representation Unlearning for Large Language Models
• Scrub It Out! Erasing Sensitive Memorization in Code Language Models via Machine Unlearning
• Module-Aware Parameter-Efficient Machine Unlearning on Transformers
• Adaptive Machine Unlearning
• Descent-to-Delete: Gradient-Based Methods for Machine Unlearning
• zkUnlearner: A Zero-Knowledge Framework for Verifiable Unlearning with Multi-Granularity and Forgery-Resistance
• MRD-LiNet: A Novel Lightweight Hybrid CNN with Gradient-Guided Unlearning for Improved Drought Stress Identification
• ConspirED: A Dataset for Cognitive Traits of Conspiracy Theories and Large Language Model Safety
• Tackling Federated Unlearning as a Parameter Estimation Problem
• Curriculum Approximate Unlearning for Session-based Recommendation
• Towards Source-Free Machine Unlearning
• Sealing The Backdoor: Unlearning Adversarial Text Triggers In Diffusion Models Using Knowledge Distillation
• Unlearning Comparator: A Visual Analytics System for Comparative Evaluation of Machine Unlearning Methods
• Mo' Memory, Mo' Problems: Stream-Native Machine Unlearning
• Invisible Watermarks, Visible Gains: Steering Machine Unlearning with Bi-Level Watermarking Design
• Membership Inference Attacks with False Discovery Rate Control
• Integrated Influence: Data Attribution with Baseline
• WSS-CL: Weight Saliency Soft-Guided Contrastive Learning for Efficient Machine Unlearning Image Classification
• Conformal Unlearning: A New Paradigm for Unlearning in Conformal Predictors
• Graph Unlearning via Embedding Reconstruction -- A Range-Null Space Decomposition Approach
• IMU: Influence-guided Machine Unlearning
• Efficient Machine Unlearning via Influence Approximation
• LoReUn: Data Itself Implicitly Provides Cues to Improve Machine Unlearning
• Zero-Shot Machine Unlearning with Proxy Adversarial Data Generation

Related

• Machine Unlearning: Linear Filtration for Logit-based Classifiers
• Approximate Data Deletion from Machine Learning Models
• Making AI Forget You: Data Deletion in Machine Learning
• Certifiable Machine Unlearning for Linear Models
• Machine Unlearning of Features and Labels