Multi-reference Tacotron by Intercross Training for Style Disentangling,Transfer and Control in Speech Synthesis

2019/04/04 by Yanyao Bian, Bian, Yanyao, Changbin Chen +5 · 5 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Music and Audio Processing #Speech Recognition and Synthesis #Speech and Audio Processing

paper · pdf · doi:10.48550/arxiv.1904.02373

openalex publication_date 2019/04/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Speech style control and transfer techniques aim to enrich the diversity and expressiveness of synthesized speech. Existing approaches model all speech styles into one representation, lacking the ability to control a specific speech feature independently. To address this issue, we introduce a novel multi-reference structure to Tacotron and propose intercross training approach, which together ensure that each sub-encoder of the multi-reference encoder independently disentangles and controls a specific style. Experimental results show that our model is able to control and transfer desired speech styles individually.

Citations

• Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
• Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning

Cited by

• Voice Conversion for Lombard Speaking Style with Implicit and Explicit Acoustic Feature Conditioning
• Controllable Context-aware Conversational Speech Synthesis
• Multi-Reference Neural TTS Stylization with Adversarial Cycle Consistency
• Improving Performance of Seen and Unseen Speech Style Transfer in End-to-end Neural TTS
• Meta-Voice: Fast few-shot style transfer for expressive voice cloning using meta learning

Related

• Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis
• Multi-speaker Multi-style Text-to-speech Synthesis With Single-speaker Single-style Training Data Scenarios
• Multi-Speaker Multi-Style Speech Synthesis with Timbre and Style Disentanglement
• Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with\n Tacotron
• Towards Multi-Scale Style Control for Expressive Speech Synthesis