Multitask Learning with Low-Level Auxiliary Tasks for Encoder-Decoder Based Speech Recognition

2017/04/05 by Toshniwal, Shubham, Tang, Hao, Lu, Liang +1 · 1 citation
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.1704.01631

Abstract

End-to-end training of deep learning-based models allows for implicit learning of intermediate representations based on the final task loss. However, the end-to-end approach ignores the useful domain knowledge encoded in explicit intermediate-level supervision. We hypothesize that using intermediate representations as auxiliary supervision at lower levels of deep networks may be a good way of combining the advantages of end-to-end training and more traditional pipeline approaches. We present experiments on conversational speech recognition where we use lower-level tasks, such as phoneme recognition, in a multitask training approach with an encoder-decoder model for direct character transcription. We compare multiple types of lower-level tasks and analyze the effects of the auxiliary tasks. Our results on the Switchboard corpus show that this approach improves recognition accuracy over a standard encoder-decoder model on the Eval2000 test set.

Cited by

• Generating Auxiliary Tasks with Reinforcement Learning

Related

• Hierarchical Multitask Learning for CTC-based Speech Recognition
• Hierarchical Multi Task Learning With CTC
• Multitask Training with Text Data for End-to-End Speech Recognition
• On the Inductive Bias of Word-Character-Level Multi-Task Learning for Speech Recognition
• Learning Shared Encoding Representation for End-to-End Speech Recognition Models