Evaluating and Improving the Robustness of Speech Command Recognition Models to Noise and Distribution Shifts

2025/07/30 by Anaïs Baranger, Baranger, Anaïs, Lucas Maison +1
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Speech Recognition and Synthesis #Speech and Audio Processing

paper · pdf · doi:10.48550/arxiv.2507.23128

openalex publication_date 2025/07/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Although prior work in computer vision has shown strong correlations between in-distribution (ID) and out-of-distribution (OOD) accuracies, such relationships remain underexplored in audio-based models. In this study, we investigate how training conditions and input features affect the robustness and generalization abilities of spoken keyword classifiers under OOD conditions. We benchmark several neural architectures across a variety of evaluation sets. To quantify the impact of noise on generalization, we make use of two metrics: Fairness (F), which measures overall accuracy gains compared to a baseline model, and Robustness (R), which assesses the convergence between ID and OOD performance. Our results suggest that noise-aware training improves robustness in some configurations. These findings shed new light on the benefits and limitations of noise-based augmentation for generalization in speech models.

Citations

• Open-Source Conversational AI with SpeechBrain 1.0
• Investigating the 'Autoencoder Behavior' in Speech Self-Supervised Models: a focus on HuBERT's Pretraining
• Speech Robust Bench: A Robustness Benchmark For Speech Recognition
• OOD-Speech: A Large Bengali Speech Recognition Dataset for Out-of-Distribution Benchmarking
• Accuracy on the Curve: On the Nonlinear Correlation of ML Performance Between Data Subpopulations
• Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift
• Accuracy on the Line: On the Strong Correlation Between\n Out-of-Distribution and In-Distribution Generalization
• HuBERT: Self-Supervised Speech Representation Learning by Masked\n Prediction of Hidden Units
• PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition
• Common Voice: A Massively-Multilingual Speech Corpus
• Google Speech Commands-Musan test set
• MUSAN: A Music, Speech, and Noise Corpus
• Adam: A Method for Stochastic Optimization
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches

Related

• PatchDSU: Uncertainty Modeling for Out of Distribution Generalization in Keyword Spotting
• Improving Noise Robustness of an End-to-End Neural Model for Automatic Speech Recognition
• Analyzing Robustness of End-to-End Neural Models for Automatic Speech Recognition
• Rethinking Evaluation in ASR: Are Our Models Robust Enough?
• Robustness of Neural Architectures for Audio Event Detection