2021/04/05 by Yangyang Shi, Shi, Yangyang, Varun Nagaraja +21 · 1 citation
Computer Science · #Algorithm #Computation and Language (cs.CL) #Computer network #Computer science #Decoding methods #Dropout (neural networks) #Encoder #FOS: Computer and information sciences #Latency (audio) #Low latency (capital markets) #Machine learning #Music and Audio Processing #Real-time computing #Speech Recognition and Synthesis #Speech and Audio Processing #Speech recognition #Telecommunications #Utterance #cs.CL
paper · pdf · doi:10.48550/arxiv.2104.02176
published in arXiv (Cornell University) (Cornell University) · 5 pages, 2 figures, submitted Interspeech 2021
arxiv created 2021/04/05 · openalex publication_date 2021/04/05 · arxiv updated 2021/04/07 · openalex created_date 2021/04/13 · openalex updated_date 2026/08/08
We propose a dynamic encoder transducer (DET) for on-device speech recognition. One DET model scales to multiple devices with different computation capacities without retraining or finetuning. To trading off accuracy and latency, DET assigns different encoders to decode different parts of an utterance. We apply and compare the layer dropout and the collaborative learning for DET training. The layer dropout method that randomly drops out encoder layers in the training phase, can do on-demand layer dropout in decoding. Collaborative learning jointly trains multiple encoders with different depths in one single model. Experiment results on Librispeech and in-house data show that DET provides a flexible accuracy and latency trade-off. Results on Librispeech show that the full-size encoder in DET relatively reduces the word error rate of the same size baseline by over 8%. The lightweight encoder in DET trained with collaborative learning reduces the model size by 25% but still gets similar WER as the full-size baseline. DET gets similar accuracy as a baseline model with better latency on a large in-house data set by assigning a lightweight encoder for the beginning part of one utterance and a full-size encoder for the rest.