TTS Skins: Speaker Conversion via ASR

2019/04/18 by Polyak, Adam, Wolf, Lior, Taigman, Yaniv · 1 citation
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Sound (cs.SD)

paper · doi:10.48550/arxiv.1904.08983

Abstract

We present a fully convolutional wav-to-wav network for converting between speakers' voices, without relying on text. Our network is based on an encoder-decoder architecture, where the encoder is pre-trained for the task of Automatic Speech Recognition, and a multi-speaker waveform decoder is trained to reconstruct the original signal in an autoregressive manner. We train the network on narrated audiobooks, and demonstrate multi-voice TTS in those voices, by converting the voice of a TTS robot.

Cited by

• Towards Generalizable Deepfake Detection via Forgery-aware Audio-Visual Adaptation: A Variational Bayesian Approach

Related

• Voice Conversion by Cascading Automatic Speech Recognition and Text-to-Speech Synthesis with Prosody Transfer
• Transfer Learning from Speech Synthesis to Voice Conversion with\n Non-Parallel Training Data
• ConVoice: Real-Time Zero-Shot Voice Style Transfer with Convolutional\n Network
• Cotatron: Transcription-Guided Speech Encoder for Any-to-Many Voice\n Conversion without Parallel Data
• The Sequence-to-Sequence Baseline for the Voice Conversion Challenge 2020: Cascading ASR and TTS