Scyclone: High-Quality and Parallel-Data-Free Voice Conversion Using Spectrogram and Cycle-Consistent Adversarial Networks

2020/05/07 by Masaya Tanaka, Tanaka, Masaya, Takashi Nose +7
Computer Science · #Audio and Speech Processing (eess.AS) #FOS: Electrical engineering #Music and Audio Processing #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2005.03334

openalex publication_date 2020/05/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper proposes Scyclone, a high-quality voice conversion (VC) technique without parallel data training. Scyclone improves speech naturalness and speaker similarity of the converted speech by introducing CycleGAN-based spectrogram conversion with a simplified WaveRNN-based vocoder. In Scyclone, a linear spectrogram is used as the conversion features instead of vocoder parameters, which avoids quality degradation due to extraction errors in fundamental frequency and voiced/unvoiced parameters. The spectrogram of source and target speakers are modeled by modified CycleGAN networks, and the waveform is reconstructed using the simplified WaveRNN with a single Gaussian probability density function. The subjective experiments with completely unpaired training data show that Scyclone is significantly better than CycleGAN-VC2, one of the existing state-of-the-art parallel-data-free VC techniques.

Citations

• Adam: A Method for Stochastic Optimization
• Auto-Encoding Variational Bayes
• Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
• Instance Normalization: The Missing Ingredient for Fast Stylization
• Energy-based Generative Adversarial Network
• Spectral Normalization for Generative Adversarial Networks
• Efficient Neural Audio Synthesis
• WaveCycleGAN2: Time-domain Neural Post-filter for Speech Waveform Generation

Related

• Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networks
• CycleGAN-VC3: Examining and Improving CycleGAN-VCs for Mel-spectrogram Conversion
• Axial Residual Networks for CycleGAN-based Voice Conversion
• Baseline System of Voice Conversion Challenge 2020 with Cyclic Variational Autoencoder and Parallel WaveGAN
• Vocoder-Free Non-Parallel Conversion of Whispered Speech With Masked Cycle-Consistent Generative Adversarial Networks