Finetuning End-to-End Models for Estonian Conversational Spoken Language Translation

2024/07/04 by Tiia Sildam, Sildam, Tiia, Andra Velve +3
Computer Science · #Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Natural Language Processing Techniques #Speech Recognition and Synthesis #Speech and dialogue systems #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2407.03809

openalex publication_date 2024/07/04 · openalex created_date 2024/07/09 · openalex updated_date 2026/07/28

Abstract

This paper investigates the finetuning of end-to-end models for bidirectional Estonian-English and Estonian-Russian conversational speech-to-text translation. Due to the limited availability of speech translation data for Estonian, we created additional training data by web scraping and synthesizing data from speech recognition datasets using machine translation. We evaluated three publicly available end-to-end models: Whisper, OWSM 3.1, and SeamlessM4T. Our results indicate that fine-tuning with synthetic data enhances translation accuracy by a large margin, with SeamlessM4T matching or surpassing cascaded speech translation systems that use state-of-the-art speech recognition and machine translation models.

Related

• End-to-End Speech-to-Text Translation: A Survey
• Harnessing Indirect Training Data for End-to-End Automatic Speech Translation: Tricks of the Trade
• End-to-end Automatic Speech Recognition and Speech Translation: Integration of Speech Foundational Models and LLMs
• SeamlessM4T: Massively Multilingual & Multimodal Machine Translation
• Granary: Speech Recognition and Translation Dataset in 25 European Languages