PITS: Variational Pitch Inference without Fundamental Frequency for End-to-End Pitch-controllable TTS

2023/02/24 by Junhyeok Lee, Wonbin Jung, Lee, Junhyeok +6
Computer Science · #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2302.12391

openalex publication_date 2023/02/24 · openalex created_date 2023/03/01 · openalex updated_date 2026/07/28

Abstract

Previous pitch-controllable text-to-speech (TTS) models rely on directly modeling fundamental frequency, leading to low variance in synthesized speech. To address this issue, we propose PITS, an end-to-end pitch-controllable TTS model that utilizes variational inference to model pitch. Based on VITS, PITS incorporates the Yingram encoder, the Yingram decoder, and adversarial training of pitch-shifted synthesis to achieve pitch-controllability. Experiments demonstrate that PITS generates high-quality speech that is indistinguishable from ground truth speech and has high pitch-controllability without quality degradation. Code, audio samples, and demo are available at https://github.com/anonymous-pits/pits.

Related

• Period VITS: Variational Inference with Explicit Pitch Modeling for End-to-end Emotional Speech Synthesis
• Stochastic Pitch Prediction Improves the Diversity and Naturalness of Speech in Glow-TTS
• FastPitch: Parallel Text-to-speech with Pitch Prediction
• HAM-TTS: Hierarchical Acoustic Modeling for Token-Based Zero-Shot Text-to-Speech with Model and Data Scaling
• Enhancement of Pitch Controllability using Timbre-Preserving Pitch Augmentation in FastPitch