SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network

2021/04/05 by William Chan, Chan, William, Daniel Park +11 · 11 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Music and Audio Processing #Natural Language Processing Techniques #Speech Recognition and Synthesis #cs.CL #cs.LG

paper · pdf · doi:10.48550/arxiv.2104.02133

submitted to INTERSPEECH

openalex publication_date 2021/04/05 · openalex created_date 2021/04/13 · arxiv created 2021/04/27 · arxiv updated 2021/04/28 · openalex updated_date 2026/07/28

Abstract

We present SpeechStew, a speech recognition model that is trained on a combination of various publicly available speech recognition datasets: AMI, Broadcast News, Common Voice, LibriSpeech, Switchboard/Fisher, Tedlium, and Wall Street Journal. SpeechStew simply mixes all of these datasets together, without any special re-weighting or re-balancing of the datasets. SpeechStew achieves SoTA or near SoTA results across a variety of tasks, without the use of an external language model. Our results include 9.0% WER on AMI-IHM, 4.7% WER on Switchboard, 8.3% WER on CallHome, and 1.3% on WSJ, which significantly outperforms prior work with strong external language models. We also demonstrate that SpeechStew learns powerful transfer learning representations. We fine-tune SpeechStew on a noisy low resource speech dataset, CHiME-6. We achieve 38.9% WER without a language model, which compares to 38.6% WER to a strong HMM baseline with a language model.

Citations

• Kaldi Speech Recognition Toolkit
• Sequence Transduction with Recurrent Neural Networks
• DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
• ImageNet classification with deep convolutional neural networks
• End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures
• Scaling Laws for Neural Language Models
• CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordings
• Conformer: Convolution-augmented Transformer for Speech Recognition
• Transformer Language Models with LSTM-based Cross-utterance Information Representation

Cited by

• AgriGPT-Omni: A Unified Speech-Vision-Text Framework for Multilingual Agricultural Intelligence
• OLMoASR: Open Models and Data for Training Robust Speech Recognition Models
• CAMÕES: A Comprehensive Automatic Speech Recognition Benchmark for European Portuguese
• Revealing the Role of Audio Channels in ASR Performance Degradation
• DeCRED: Decoder-Centric Regularization for Encoder-Decoder Based Speech Recognition
• Application of Whisper in Clinical Practice: the Post-Stroke Speech Assessment during a Naming Task
• Analyzing and Fine-Tuning Whisper Models for Multilingual Pilot Speech Transcription in the Cockpit
• OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaning
• Loquacious Set: 25,000 Hours of Transcribed and Diverse English Speech Recognition Data for Research and Commercial Use
• LegoSLM: Connecting LLM with Speech Encoder using CTC Posteriors
• Large-Scale Pre-Training of End-to-End Multi-Talker ASR for Meeting Transcription with Single Distant Microphone
• SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training

Related

• Training Neural Speech Recognition Systems with Synthetic Speech Augmentation
• OLMoASR: Open Models and Data for Training Robust Speech Recognition Models
• Automatic Speech Recognition in the Modern Era: Architectures, Training, and Evaluation
• STTATTS: Unified Speech-To-Text And Text-To-Speech Model
• The CAPIO 2017 Conversational Speech Recognition System