Have you forgotten? A method to assess if machine learning models have forgotten data

2020/04/21 by Xiao Liu, Sotirios A. Tsaftaris, Liu, Xiao +1 · 1 citation
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning in Healthcare #Privacy-Preserving Technologies in Data

paper · pdf · doi:10.48550/arxiv.2004.10129

openalex publication_date 2020/04/21 · openalex created_date 2020/05/01 · openalex updated_date 2026/07/28

Abstract

In the era of deep learning, aggregation of data from several sources is a common approach to ensuring data diversity. Let us consider a scenario where several providers contribute data to a consortium for the joint development of a classification model (hereafter the target model), but, now one of the providers decides to leave. This provider requests that their data (hereafter the query dataset) be removed from the databases but also that the model `forgets' their data. In this paper, for the first time, we want to address the challenging question of whether data have been forgotten by a model. We assume knowledge of the query dataset and the distribution of a model's output. We establish statistical methods that compare the target's outputs with outputs of models trained with different datasets. We evaluate our approach on several benchmark datasets (MNIST, CIFAR-10 and SVHN) and on a cardiac pathology diagnosis task using data from the Automated Cardiac Diagnosis Challenge (ACDC). We hope to encourage studies on what information a model retains and inspire extensions in more complex settings.

Citations

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Gradient-based learning applied to document recognition
• Differential Privacy and Machine Learning: a Survey and Review
• ImageNet classification with deep convolutional neural networks
• Deep Learning Techniques for Automatic MRI Cardiac Multi-Structures Segmentation and Diagnosis: Is the Problem Solved?
• Déjà Vu: an empirical evaluation of the memorization properties of ConvNets
• BrainTorrent: A Peer-to-Peer Environment for Decentralized Federated Learning
• Learning Multiple Layers of Features from Tiny Images

Cited by

• Towards Lifecycle Unlearning Commitment Management: Measuring Sample-level Unlearning Completeness

Related

• Why patient data cannot be easily forgotten?
• Catastrophic Forgetting in the Context of Model Updates
• Making AI Forget You: Data Deletion in Machine Learning
• To impute or not to impute: How machine learning modelers treat missing data
• A Survey on Data Cleaning Methods for Improved Machine Learning Model Performance