Simultaneous Speech-to-Speech Translation System with Neural Incremental ASR, MT, and TTS

2020/11/10 by Katsuhito Sudoh, Takatomo Kano, Sudoh, Katsuhito +9 · 2 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2011.04845

openalex publication_date 2020/11/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper presents a newly developed, simultaneous neural speech-to-speech translation system and its evaluation. The system consists of three fully-incremental neural processing modules for automatic speech recognition (ASR), machine translation (MT), and text-to-speech synthesis (TTS). We investigated its overall latency in the system's Ear-Voice Span and speaking latency along with module-level performance.

Citations

• JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesis
• Simultaneous Neural Machine Translation using Connectionist Temporal Classification

Cited by

• Simulstream: Open-Source Toolkit for Evaluation and Demonstration of Streaming Speech-to-Text Translation Systems
• SimulLR: Simultaneous Lip Reading Transducer with Attention-Guided Adaptive Memory

Related

• Label-Synchronous Neural Transducer for E2E Simultaneous Speech Translation
• Large-Scale Streaming End-to-End Speech Translation with Neural Transducers
• Overcoming Latency Bottlenecks in On-Device Speech Translation: A Cascaded Approach with Alignment-Based Streaming MT
• Direct Speech-to-Speech Neural Machine Translation: A Survey
• A Technical Report: BUT Speech Translation Systems