BERT, can HE predict contrastive focus? Predicting and controlling prominence in neural TTS using a language model

2022/07/04 by Stephenson, Brooke, Besacier, Laurent, Girin, Laurent +1 · 3 citations
#Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2207.01718

Abstract

Several recent studies have tested the use of transformer language model representations to infer prosodic features for text-to-speech synthesis (TTS). While these studies have explored prosody in general, in this work, we look specifically at the prediction of contrastive focus on personal pronouns. This is a particularly challenging task as it often requires semantic, discursive and/or pragmatic knowledge to predict correctly. We collect a corpus of utterances containing contrastive focus and we evaluate the accuracy of a BERT model, finetuned to predict quantized acoustic prominence features, on these samples. We also investigate how past utterances can provide relevant information for this prediction. Furthermore, we evaluate the controllability of pronoun prominence in a TTS model conditioned on acoustic prominence features.

Cited by

• Improving French Synthetic Speech Quality via SSML Prosody Control
• StressTest: Can YOUR Speech LM Handle the Stress?
• WHISTRESS: Enriching Transcriptions with Sentence Stress Detection

Related

• Predicting Prosodic Prominence from Text with Pre-trained Contextualized Word Representations
• What does BERT learn about prosody?
• Spontaneous Speech Variables for Evaluating LLMs Cognitive Plausibility
• Emphasis control for parallel neural TTS
• Emphasis Sensitivity in Speech Representations