Disentanglement of Emotional Style and Speaker Identity for Expressive Voice Conversion

2021/10/20 by Zongyang Du, Berrak Şişman, Du, Zongyang +5 · 6 citations
Computer Science · #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2110.10326

openalex publication_date 2021/10/20 · openalex created_date 2022/07/23 · openalex updated_date 2026/08/01

Abstract

Expressive voice conversion performs identity conversion for emotional speakers by jointly converting speaker identity and emotional style. Due to the hierarchical structure of speech emotion, it is challenging to disentangle the emotional style for different speakers. Inspired by the recent success of speaker disentanglement with variational autoencoder (VAE), we propose an any-to-any expressive voice conversion framework, that is called StyleVC. StyleVC is designed to disentangle linguistic content, speaker identity, pitch, and emotional style information. We study the use of style encoder to model emotional style explicitly. At run-time, StyleVC converts both speaker identity and emotional style for arbitrary speakers. Experiments validate the effectiveness of our proposed framework in both objective and subjective evaluations.

Cited by

• NaturalVoices: A Large-Scale, Spontaneous and Emotional Podcast Dataset for Voice Conversion
• Emotional Styles Hide in Deep Speaker Embeddings: Disentangle Deep Speaker Embeddings for Speaker Clustering
• Enhancing In-the-Wild Speech Emotion Conversion with Resynthesis-based Duration Modeling
• Maestro-EVC: Controllable Emotional Voice Conversion Guided by References and Explicit Prosody
• MemoryTalker: Personalized Speech-Driven 3D Facial Animation via Audio-Guided Stylization
• Fast-VGAN: Lightweight Voice Conversion with Explicit Control of F0 and Duration Parameters

Related

• Expressive Voice Conversion: A Joint Framework for Speaker Identity and Emotional Style Transfer
• Converting Anyone's Voice: End-to-End Expressive Voice Conversion with a Conditional Diffusion Model
• Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset
• Towards Better Disentanglement in Non-Autoregressive Zero-Shot Expressive Voice Conversion
• Limited Data Emotional Voice Conversion Leveraging Text-to-Speech: Two-stage Sequence-to-Sequence Training