Certifying the Right to Be Forgotten: Primal-Dual Optimization for Sample and Label Unlearning in Vertical Federated Learning

2025/12/29 by Yu Jiang, Jiang, Yu, Xindi Tong +9
Computer Science · Social Sciences · #Cryptography and Security (cs.CR) #Domain Adaptation and Few-Shot Learning #Ethics and Social Impacts of AI #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy-Preserving Technologies in Data

paper · doi:10.48550/arxiv.2512.23171

openalex publication_date 2025/12/29 · openalex created_date 2025/12/31 · openalex updated_date 2026/07/28

Abstract

Federated unlearning has become an attractive approach to address privacy concerns in collaborative machine learning, for situations when sensitive data is remembered by AI models during the machine learning process. It enables the removal of specific data influences from trained models, aligning with the growing emphasis on the "right to be forgotten." While extensively studied in horizontal federated learning, unlearning in vertical federated learning (VFL) remains challenging due to the distributed feature architecture. VFL unlearning includes sample unlearning that removes specific data points' influence and label unlearning that removes entire classes. Since different parties hold complementary features of the same samples, unlearning tasks require cross-party coordination, creating computational overhead and complexities from feature interdependencies. To address such challenges, we propose FedORA (Federated Optimization for data Removal via primal-dual Algorithm), designed for sample and label unlearning in VFL. FedORA formulates the removal of certain samples or labels as a constrained optimization problem solved using a primal-dual framework. Our approach introduces a new unlearning loss function that promotes classification uncertainty rather than misclassification. An adaptive step size enhances stability, while an asymmetric batch design, considering the prior influence of the remaining data on the model, handles unlearning and retained data differently to efficiently reduce computational costs. We provide theoretical analysis proving that the model difference between FedORA and Train-from-scratch is bounded, establishing guarantees for unlearning effectiveness. Experiments on tabular and image datasets demonstrate that FedORA achieves unlearning effectiveness and utility preservation comparable to Retrain with reduced computation and communication overhead.

Citations

• HSplitLoRA: A Heterogeneous Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
• Forgetting Any Data at Any Time: A Theoretically Certified Unlearning Framework for Vertical Federated Learning
• Unlearning Clients, Features and Samples in Vertical Federated Learning
• Vertical Federated Unlearning via Backdoor Certification
• FedUHB: Accelerating Federated Unlearning via Polyak Heavy Ball Method
• Efficient Federated Unlearning with Adaptive Differential Privacy Preservation
• A few-shot Label Unlearning in Vertical Federated Learning
• SplitLoRA: A Split Parameter-Efficient Fine-Tuning Framework for Large Language Models
• Guaranteeing Data Privacy in Federated Unlearning with Dynamic User Participation
• Privacy-Preserving Federated Unlearning with Certified Client Removal
• Communication Efficient and Provable Federated Unlearning
• A Survey on Federated Unlearning: Challenges, Methods, and Future Directions
• Class-wise Federated Unlearning: Harnessing Active Forgetting with Teacher-Student Memory Generation
• FedRecover: Recovering from Poisoning Attacks in Federated Learning using Historical Information
• Federated Unlearning: How to Efficiently Erase a Client in FL?
• Communication Acceleration of Local Gradient Methods via an Accelerated Primal-Dual Algorithm with Inexact Prox
• Large-Scale Methods for Distributionally Robust Optimization
• Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks
• Machine Unlearning
• CoCoA: A General Framework for Communication-Efficient Distributed Optimization
• Membership Inference Attacks against Machine Learning Models
• Densely Connected Convolutional Networks
• Optimization Methods for Large-Scale Machine Learning
• Optimization Methods for Large-Scale Machine Learning
• Deep Residual Learning for Image Recognition
• An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks

Related

• Forgetting Any Data at Any Time: A Theoretically Certified Unlearning Framework for Vertical Federated Learning
• FedUHB: Accelerating Federated Unlearning via Polyak Heavy Ball Method
• Vertical Federated Unlearning via Backdoor Certification
• A few-shot Label Unlearning in Vertical Federated Learning
• A Survey on Federated Unlearning: Challenges, Methods, and Future Directions