2019/01/01 by Matthias Sperber, Sperber, Matthias, Graham Neubig +5 · 2 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling
paper · pdf · doi:10.48550/arxiv.1904.07209
openalex publication_date 2019/04/15 · openalex created_date 2022/07/24 · openalex updated_date 2026/07/28
Speech translation has traditionally been approached through cascaded models\nconsisting of a speech recognizer trained on a corpus of transcribed speech,\nand a machine translation system trained on parallel texts. Several recent\nworks have shown the feasibility of collapsing the cascade into a single,\ndirect model that can be trained in an end-to-end fashion on a corpus of\ntranslated speech. However, experiments are inconclusive on whether the cascade\nor the direct model is stronger, and have only been conducted under the\nunrealistic assumption that both are trained on equal amounts of data, ignoring\nother available speech recognition and machine translation corpora.\n In this paper, we demonstrate that direct speech translation models require\nmore data to perform well than cascaded models, and while they allow including\nauxiliary data through multi-task training, they are poor at exploiting such\ndata, putting them at a severe disadvantage. As a remedy, we propose the use of\nend-to-end trainable models with two attention mechanisms, the first\nestablishing source speech to source text alignments, the second modeling\nsource to target text alignment. We show that such models naturally decompose\ninto multi-task-trainable recognition and translation tasks and propose an\nattention-passing technique that alleviates error propagation issues in a\nprevious formulation of a model with two attention stages. Our proposed model\noutperforms all examined baselines and is able to exploit auxiliary training\ndata much more effectively than direct attentional models.\n