Towards Machine Unlearning for Paralinguistic Speech Processing

2025/06/02 by Orchid Chetia Phukan, Phukan, Orchid Chetia, Girish +14 · 1 citation
Computer Science · #Speech Recognition and Synthesis

paper · pdf · doi:10.48550/arxiv.2506.02230

Abstract

In this work, we pioneer the study of Machine Unlearning (MU) for Paralinguistic Speech Processing (PSP). We focus on two key PSP tasks: Speech Emotion Recognition (SER) and Depression Detection (DD). To this end, we propose, SISA++, a novel extension to previous state-of-the-art (SOTA) MU method, SISA by merging models trained on different shards with weight-averaging. With such modifications, we show that SISA++ preserves performance more in comparison to SISA after unlearning in benchmark SER (CREMA-D) and DD (E-DAIC) datasets. Also, to guide future research for easier adoption of MU for PSP, we present ``cookbook recipes'' - actionable recommendations for selecting optimal feature representations and downstream architectures that can mitigate performance degradation after the unlearning process.

Citations

• Machine Unlearning for Image-to-Image Generative Models
• Fast Model Debias with Machine Unlearning
• KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment
• Weight Averaging: A Simple Yet Effective Method to Overcome Catastrophic Forgetting in Automatic Speech Recognition
• A Survey of Machine Unlearning
• Making Recommender Systems Forget: Learning and Unlearning for Erasable Recommendation
• Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
• Privacy-preserving Speech Emotion Recognition through Semi-Supervised Federated Learning
• XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale
• WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
• Towards Probabilistic Verification of Machine Unlearning
• x-vectors meet emotions: A study on dependencies between emotion and\n speaker recognition
• Attention Is All You Need
• Understanding Black-box Predictions via Influence Functions
• Membership Inference Attacks against Machine Learning Models

Cited by

• Machine Unlearning in Speech Emotion Recognition via Forget Set Alone

Related

• ParaLBench: A Large-Scale Benchmark for Computational Paralinguistics over Acoustic Foundation Models
• Rethinking Cross-Corpus Speech Emotion Recognition Benchmarking: Are Paralinguistic Pre-Trained Representations Sufficient?
• Are Paralinguistic Representations all that is needed for Speech Emotion Recognition?
• SeQuiFi: Mitigating Catastrophic Forgetting in Speech Emotion Recognition with Sequential Class-Finetuning
• Learning to detect dysarthria from raw speech