EMO-Debias: Benchmarking Gender Debiasing Techniques in Multi-Label Speech Emotion Recognition

2025/06/05 by Yi‐Cheng Lin, Lin, Yi-Cheng, Huang-Cheng Chou +4 · 2 citations
Computer Science · Psychology · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #Emotion and Mood Recognition #FOS: Computer and information sciences #FOS: Electrical engineering #Hate Speech and Cyberbullying Detection #Sentiment Analysis and Opinion Mining #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2506.04652

openalex publication_date 2025/06/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Speech emotion recognition (SER) systems often exhibit gender bias. However, the effectiveness and robustness of existing debiasing methods in such multi-label scenarios remain underexplored. To address this gap, we present EMO-Debias, a large-scale comparison of 13 debiasing methods applied to multi-label SER. Our study encompasses techniques from pre-processing, regularization, adversarial learning, biased learners, and distributionally robust optimization. Experiments conducted on acted and naturalistic emotion datasets, using WavLM and XLSR representations, evaluate each method under conditions of gender imbalance. Our analysis quantifies the trade-offs between fairness and accuracy, identifying which approaches consistently reduce gender performance gaps without compromising overall model performance. The findings provide actionable insights for selecting effective debiasing strategies and highlight the impact of dataset distributions.

Citations

• Mitigating Subgroup Disparities in Multi-Label Speech Emotion Recognition: A Pseudo-Labeling and Unsupervised Learning Approach
• Is It Still Fair? Investigating Gender Fairness in Cross-Corpus Speech Emotion Recognition
• Unlabeled Debiasing in Downstream Tasks via Class-wise Low Variance Regularization
• Beyond the binary: Limitations and possibilities of gender-related speech technology research
• Beyond The Binary: Limitations and Possibilities of Gender-Related Speech Technology Research
• Improving Speech Emotion Recognition in Under-Resourced Languages via Speech-to-Speech Translation with Bootstrapping Data Selection
• Stimulus Modality Matters: Impact of Perceptual Evaluations from Different Modalities on Speech Emotion Recognition System Performance
• Spoken Stereoset: On Evaluating Social Bias Toward Speaker in Speech Large Language Models
• Listen and Speak Fairly: A Study on Semantic Gender Bias in Speech Integrated Large Language Models
• A Large-Scale Evaluation of Speech Foundation Models
• EMO-SUPERB: An In-depth Look at Speech Emotion Recognition
• Exploring data augmentation in bias mitigation against non-native-accented speech
• Soft-prompt Tuning for Large Language Models to Evaluate Bias
• Signal Is Harder To Learn Than Bias: Debiasing with Focal Loss
• BLIND: Bias Removal With No Demographics
• Speech Emotion Recognition using Supervised Deep Recurrent System for Mental Health Monitoring
• Learning Debiased Representation via Disentangled Feature Augmentation
• Diverse Adversaries for Mitigating Bias in Training
• Learning from Failure: Training Debiased Classifier from Biased Classifier
• Common Voice: A Massively-Multilingual Speech Corpus
• Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization
• Class-Balanced Loss Based on Effective Number of Samples
• Adversarial Removal of Demographic Attributes from Text Data
• Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels
• Towards Robust and Privacy-preserving Text Representations
• Equality of Opportunity in Supervised Learning

Cited by

• TAU: A Benchmark for Cultural Sound Understanding Beyond Semantics
• MI-Fuse: Label Fusion for Unsupervised Domain Adaptation with Closed-Source Large-Audio Language Model

Related

• Is It Still Fair? Investigating Gender Fairness in Cross-Corpus Speech Emotion Recognition
• A Multibias-mitigated and Sentiment Knowledge Enriched Transformer for Debiasing in Multimodal Conversational Emotion Recognition
• CO-VADA: A Confidence-Oriented Voice Augmentation Debiasing Approach for Fair Speech Emotion Recognition
• Gender Bias in Emotion Recognition by Large Language Models
• GEmo-CLAP: Gender-Attribute-Enhanced Contrastive Language-Audio Pretraining for Accurate Speech Emotion Recognition