2021/02/05 by Ruizhi Li, Li, Ruizhi, Gregory Sell +3
Computer Science · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2102.03055
openalex publication_date 2021/02/05 · openalex created_date 2022/03/05 · openalex updated_date 2026/07/28
Performance degradation of an Automatic Speech Recognition (ASR) system is\ncommonly observed when the test acoustic condition is different from training.\nHence, it is essential to make ASR systems robust against various environmental\ndistortions, such as background noises and reverberations. In a multi-stream\nparadigm, improving robustness takes account of handling a variety of unseen\nsingle-stream conditions and inter-stream dynamics. Previously, a practical\ntwo-stage training strategy was proposed within multi-stream end-to-end ASR,\nwhere Stage-2 formulates the multi-stream model with features from Stage-1\nUniversal Feature Extractor (UFE). In this paper, as an extension, we introduce\na two-stage augmentation scheme focusing on mismatch scenarios: Stage-1\nAugmentation aims to address single-stream input varieties with data\naugmentation techniques; Stage-2 Time Masking applies temporal masks on UFE\nfeatures of randomly selected streams to simulate diverse stream combinations.\nDuring inference, we also present adaptive Connectionist Temporal\nClassification (CTC) fusion with the help of hierarchical attention mechanisms.\nExperiments have been conducted on two datasets, DIRHA and AMI, as a\nmulti-stream scenario. Compared with the previous training strategy,\nsubstantial improvements are reported with relative word error rate reductions\nof 29.7-59.3% across several unseen stream combinations.\n