Population Based Training for Data Augmentation and Regularization in\n Speech Recognition

2020/10/08 by Daniel Haziza, Jérémy Rapin, Haziza, Daniel +3
Computer Science · #Speech Recognition and Synthesis #Machine Learning and Data Classification #Music and Audio Processing

paper · pdf · doi:10.48550/arxiv.2010.03899

Abstract

Varying data augmentation policies and regularization over the course of\noptimization has led to performance improvements over using fixed values. We\nshow that population based training is a useful tool to continuously search\nthose hyperparameters, within a fixed budget. This greatly simplifies the\nexperimental burden and computational cost of finding such optimal schedules.\nWe experiment in speech recognition by optimizing SpecAugment this way, as well\nas dropout. It compares favorably to a baseline that does not change those\nhyperparameters over the course of training, with an 8% relative WER\nimprovement. We obtain 5.18% word error rate on LibriSpeech's test-other.\n

Related

• Does Speech enhancement of publicly available data help build robust Speech Recognition Systems?
• O-1: Self-training with Oracle and 1-best Hypothesis
• Macro-block dropout for improved regularization in training end-to-end speech recognition models
• Improved Regularization Techniques for End-to-End Speech Recognition
• Data Augmentation and Hyperparameter Tuning for Low-Resource MFA