2023/06/20 by Swiatkowski, Jakub, Wang, Duo, Babianski, Mikolaj +5
#Audio and Speech Processing (eess.AS) #FOS: Electrical engineering #electronic engineering #information engineering
paper · doi:10.48550/arxiv.2306.11662
Speech generation for machine dubbing adds complexity to conventional Text-To-Speech solutions as the generated output is required to match the expressiveness, emotion and speaking rate of the source content. Capturing and transferring details and variations in prosody is a challenge. We introduce phrase-level cross-lingual prosody transfer for expressive multi-lingual machine dubbing. The proposed phrase-level prosody transfer delivers a significant 6.2% MUSHRA score increase over a baseline with utterance-level global prosody transfer, thereby closing the gap between the baseline and expressive human dubbing by 23.2%, while preserving intelligibility of the synthesised speech.