Towards Robust Knowledge Removal in Federated Learning with High Data Heterogeneity

2025/10/15 by Riccardo Delli Santi, Santi, Riccardo, Riccardo Salami +3
Computer Science · Neuroscience · #Brain Tumor Detection and Classification #FOS: Computer and information sciences #Machine Learning (cs.LG) #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2510.13606

openalex publication_date 2025/10/15 · openalex created_date 2025/10/17 · openalex updated_date 2026/07/28

Abstract

Nowdays, there are an abundance of portable devices capable of collecting large amounts of data and with decent computational power. This opened the possibility to train AI models in a distributed manner, preserving the participating clients' privacy. However, because of privacy regulations and safety requirements, elimination upon necessity of a client contribution to the model has become mandatory. The cleansing process must satisfy specific efficacy and time requirements. In recent years, research efforts have produced several knowledge removal methods, but these require multiple communication rounds between the data holders and the process coordinator. This can cause the unavailability of an effective model up to the end of the removal process, which can result in a disservice to the system users. In this paper, we introduce an innovative solution based on Task Arithmetic and the Neural Tangent Kernel, to rapidly remove a client's influence from a model.

Citations

• Update Your Transformer to the Latest Release: Re-Basin of Task Vectors
• Closed-form merging of parameter-efficient modules for Federated Continual Learning
• CLIP with Generative Latent Replay: a Strong Baseline for Incremental Learning
• Federated Class-Incremental Learning with Hierarchical Generative Prototypes
• A Second-Order Perspective on Model Compositionality and Incremental Learning
• Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivity
• Semantic Residual Prompts for Continual Learning
• GradMA: A Gradient-Memory-based Accelerated Federated Learning with Alleviated Catastrophic Forgetting
• Editing Models with Task Arithmetic
• FedRecover: Recovering from Poisoning Attacks in Federated Learning using Historical Information
• FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction
• Merging Models with Fisher-Weighted Averaging
• FedProto: Federated Prototype Learning across Heterogeneous Clients
• Learning Transferable Visual Models From Natural Language Supervision
• Federated Optimization in Heterogeneous Networks

Related

• Enhancing Heterogeneous Federated Learning with Knowledge Extraction and Multi-Model Fusion
• Federated Unlearning with Knowledge Distillation
• Neural Tangent Kernel Empowered Federated Learning
• Privacy Preserving Stochastic Channel-Based Federated Learning with Neural Network Pruning
• Machine unlearning via GAN