Controllable Machine Unlearning via Gradient Pivoting

2025/10/22 by Youngsik Hwang, Hwang, Youngsik, Dong‐Young Lim +1
Computer Science · #Advanced Multi-Objective Optimization Algorithms #Advanced Neural Network Applications #FOS: Computer and information sciences #FOS: Mathematics #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Optimization and Control (math.OC)

paper · pdf · doi:10.48550/arxiv.2510.19226

openalex publication_date 2025/10/22 · openalex created_date 2025/10/24 · openalex updated_date 2026/07/28

Abstract

Machine unlearning (MU) aims to remove the influence of specific data from a trained model. However, approximate unlearning methods, often formulated as a single-objective optimization (SOO) problem, face a critical trade-off between unlearning efficacy and model fidelity. This leads to three primary challenges: the risk of over-forgetting, a lack of fine-grained control over the unlearning process, and the absence of metrics to holistically evaluate the trade-off. To address these issues, we reframe MU as a multi-objective optimization (MOO) problem. We then introduce a novel algorithm, Controllable Unlearning by Pivoting Gradient (CUP), which features a unique pivoting mechanism. Unlike traditional MOO methods that converge to a single solution, CUP's mechanism is designed to controllably navigate the entire Pareto frontier. This navigation is governed by a single intuitive hyperparameter, the `unlearning intensity', which allows for precise selection of a desired trade-off. To evaluate this capability, we adopt the hypervolume indicator, a metric that captures both the quality and diversity of the entire set of solutions an algorithm can generate. Our experimental results demonstrate that CUP produces a superior set of Pareto-optimal solutions, consistently outperforming existing methods across various vision tasks.

Citations

• A Closer Look at Machine Unlearning for Large Language Models
• Dual Cone Gradient Descent for Training Physics-Informed Neural Networks
• Adversarial Machine Unlearning
• Machine Unlearning via Null Space Calibration
• Machine Unlearning for Image-to-Image Generative Models
• Layer Attack Unlearning: Fast and Accurate Machine Unlearning via Layer Level Attack and Knowledge Distillation
• Learn to Unlearn for Deep Neural Networks: Minimizing Unlearning Interference with Gradient Projection
• SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation
• Fast Model Debias with Machine Unlearning
• Fair Machine Unlearning: Data Removal while Mitigating Disparities
• Gradient Surgery for One-shot Unlearning on Generative Model
• Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Models
• Model Sparsity Can Simplify Machine Unlearning
• Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models
• Ablating Concepts in Text-to-Image Diffusion Models
• Boundary Unlearning
• Erasing Concepts from Diffusion Models
• Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers
• Hidden Poison: Machine Unlearning Enables Camouflaged Poisoning Attacks
• Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• Unrolling SGD: Understanding Factors Influencing Machine Unlearning
• Hard to Forget: Poisoning Attacks on Certified Machine Unlearning
• Denoising Diffusion Probabilistic Models
• Approximate Data Deletion from Machine Learning Models
• Gradient Surgery for Multi-Task Learning
• Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep\n Networks
• Making AI Forget You: Data Deletion in Machine Learning
• Deep Residual Learning for Image Recognition
• U-Net: Convolutional Networks for Biomedical Image Segmentation

Related

• Improving Unlearning with Model Updates Probably Aligned with Gradients
• Machine Unlearning under Overparameterization
• ∇ τ: Gradient-based and Task-Agnostic machine Unlearning
• Targeted Unlearning with Single Layer Unlearning Gradient
• Learning to Unlearn while Retaining: Combating Gradient Conflicts in Machine Unlearning