FastPitch: Parallel Text-to-speech with Pitch Prediction

2020/06/11 by Łańcucki, Adrian · 10 citations
#Audio and Speech Processing (eess.AS) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Sound (cs.SD) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2006.06873

Abstract

We present FastPitch, a fully-parallel text-to-speech model based on FastSpeech, conditioned on fundamental frequency contours. The model predicts pitch contours during inference. By altering these predictions, the generated speech can be more expressive, better match the semantic of the utterance, and in the end more engaging to the listener. Uniformly increasing or decreasing pitch with FastPitch generates speech that resembles the voluntary modulation of voice. Conditioning on frequency contours improves the overall quality of synthesized speech, making it comparable to state-of-the-art. It does not introduce an overhead, and FastPitch retains the favorable, fully-parallel Transformer architecture, with over 900x real-time factor for mel-spectrogram synthesis of a typical utterance.

Cited by

• Pseudo-Cepstrum: Pitch Modification for Mel-Based Neural Vocoders
• F5-TTS-RO: Extending F5-TTS to Romanian TTS via Lightweight Input Adaptation
• Arabic TTS with FastPitch: Reproducible Baselines, Adversarial Training, and Oversmoothing Analysis
• Audio Palette: A Diffusion Transformer with Multi-Signal Conditioning for Controllable Foley Synthesis
• HuLA: Prosody-Aware Anti-Spoofing with Multi-Task Learning for Expressive and Emotional Synthetic Speech
• SEA-Spoof: Bridging The Gap in Multilingual Audio Deepfake Detection for South-East Asian
• Accelerating Diffusion Transformer-Based Text-to-Speech with Transformer Layer Caching
• Facilitating Personalized TTS for Dysarthric Speakers Using Knowledge Anchoring and Curriculum Learning
• MultiGen: Child-Friendly Multilingual Speech Generator with LLMs
• Adaptive Duration Model for Text Speech Alignment

Related

• Incremental FastPitch: Chunk-based High Quality Text to Speech
• Enhancement of Pitch Controllability using Timbre-Preserving Pitch Augmentation in FastPitch
• FastPitchFormant: Source-filter based Decomposed Modeling for Speech Synthesis
• FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
• FastSpeech: Fast, Robust and Controllable Text to Speech