"Alexa, can you forget me?" Machine Unlearning Benchmark in Spoken Language Understanding

2025/05/21 by Alkis Koudounas, Claudio Savelli, Koudounas, Alkis +5 · 3 citations
Computer Science · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Sound (cs.SD) #Speech and dialogue systems #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2505.15700

openalex publication_date 2025/05/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/03

Abstract

Machine unlearning, the process of efficiently removing specific information from machine learning models, is a growing area of interest for responsible AI. However, few studies have explored the effectiveness of unlearning methods on complex tasks, particularly speech-related ones. This paper introduces UnSLU-BENCH, the first benchmark for machine unlearning in spoken language understanding (SLU), focusing on four datasets spanning four languages. We address the unlearning of data from specific speakers as a way to evaluate the quality of potential "right to be forgotten" requests. We assess eight unlearning techniques and propose a novel metric to simultaneously better capture their efficacy, utility, and efficiency. UnSLU-BENCH sets a foundation for unlearning in SLU and reveals significant differences in the effectiveness and computational feasibility of various techniques.

Citations

• Speech-MASSIVE: A Multilingual Speech Dataset for SLU and Beyond
• Machine Unlearning in Generative AI: A Survey
• RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models
• Gone but Not Forgotten: Improved Benchmarks for Machine Unlearning
• From Speech to Data: Unraveling Google's Use of Voice Data for User Profiling
• Inexact Unlearning Needs More Careful Evaluations to Avoid a False Sense of Privacy
• Machine Unlearning for Image-to-Image Generative Models
• TOFU: A Task of Fictitious Unlearning for LLMs
• Towards Machine Unlearning Benchmarks: Forgetting the Personal Identities in Facial Recognition Systems
• Fast Model Debias with Machine Unlearning
• Who's Harry Potter? Approximate Unlearning in LLMs
• Towards Unbounded Machine Unlearning
• Knowledge Unlearning for Mitigating Privacy Risks in Language Models
• Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher
• Towards Adversarial Evaluations for Inexact Machine Unlearning
• XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale
• HuBERT: Self-Supervised Speech Representation Learning by Masked\n Prediction of Hidden Units
• SLURP: A Spoken Language Understanding Resource Package
• Unsupervised Cross-lingual Representation Learning for Speech\n Recognition
• wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
• Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep\n Networks
• Speech Model Pre-training for End-to-End Spoken Language Understanding

Cited by

• FAME: Fictional Actors for Multilingual Erasure
• Protecting Bystander Privacy via Selective Hearing in Audio LLMs
• Machine Unlearning in Speech Emotion Recognition via Forget Set Alone

Related

• Machine Unlearning for Traditional Models and Large Language Models: A Short Survey
• Machine Unlearning of Pre-trained Large Language Models
• A Comprehensive Survey of Machine Unlearning Techniques for Large Language Models
• Selective Forgetting: Advancing Machine Unlearning Techniques and Evaluation in Language Models
• Does Machine Unlearning Truly Remove Knowledge?