HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks

2020/06/10 by Jiaqi Su, Su, Jiaqi, Zeyu Jin +3 · 9 citations
Computer Science · #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2006.05694

openalex publication_date 2020/06/10 · openalex created_date 2020/06/19 · openalex updated_date 2026/07/28

Abstract

Real-world audio recordings are often degraded by factors such as noise, reverberation, and equalization distortion. This paper introduces HiFi-GAN, a deep learning method to transform recorded speech to sound as though it had been recorded in a studio. We use an end-to-end feed-forward WaveNet architecture, trained with multi-scale adversarial discriminators in both the time domain and the time-frequency domain. It relies on the deep feature matching losses of the discriminators to improve the perceptual quality of enhanced speech. The proposed model generalizes well to new speakers, new speech content, and new environments. It significantly outperforms state-of-the-art baseline methods in both objective and subjective experiments.

Citations

• Speech enhancement using a minimum mean-square error log-spectral amplitude estimator
• NIPS 2016 Tutorial: Generative Adversarial Networks
• MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis

Cited by

• Aliasing-Free Neural Audio Synthesis
• Generative adversarial networks for speech processing: A review
• Moises-Light: Resource-efficient Band-split U-Net For Music Source Separation
• TF-Restormer: Complex Spectral Prediction for Speech Restoration
• NORESQA: A Framework for Speech Quality Assessment using Non-Matching References
• SaD: A Scenario-Aware Discriminator for Speech Enhancement
• Leveraging Discriminative Latent Representations for Conditioning GAN-Based Speech Enhancement
• Enhancing Low-Quality Voice Recordings Using Disentangled Channel Factor and Neural Waveform Model
• Universal Speech Enhancement with Regression and Generative Mamba

Related

• HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
• FINALLY: fast and universal speech enhancement with studio-like quality
• HiFi-SR: A Unified Generative Transformer-Convolutional Adversarial Network for High-Fidelity Speech Super-Resolution
• Source-Filter HiFi-GAN: Fast and Pitch Controllable High-Fidelity Neural Vocoder
• EVA-GAN: Enhanced Various Audio Generation via Scalable Generative Adversarial Networks