Diff-SV: A Unified Hierarchical Framework for Noise-Robust Speaker Verification Using Score-Based Diffusion Probabilistic Models

2023/09/14 by Ju-ho Kim, Jungwoo Heo, Kim, Ju-ho +7 · 4 citations
Computer Science · #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Music and Audio Processing #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2309.08320

openalex publication_date 2023/09/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Background noise considerably reduces the accuracy and reliability of speaker verification (SV) systems. These challenges can be addressed using a speech enhancement system as a front-end module. Recently, diffusion probabilistic models (DPMs) have exhibited remarkable noise-compensation capabilities in the speech enhancement domain. Building on this success, we propose Diff-SV, a noise-robust SV framework that leverages DPM. Diff-SV unifies a DPM-based speech enhancement system with a speaker embedding extractor, and yields a discriminative and noise-tolerable speaker representation through a hierarchical structure. The proposed model was evaluated under both in-domain and out-of-domain noisy conditions using the VoxCeleb1 test set, an external noise source, and the VOiCES corpus. The obtained experimental results demonstrate that Diff-SV achieves state-of-the-art performance, outperforming recently proposed noise-robust SV systems.

Cited by

• Noise-Conditioned Mixture-of-Experts Framework for Robust Speaker Verification
• A Stage-Wise Learning Strategy with Fixed Anchors for Robust Speaker Verification
• ParaNoise-SV: Integrated Approach for Noise-Robust Speaker Verification with Parallel Joint Learning of Speech Enhancement and Noise Extraction
• A Data-Driven Diffusion-based Approach for Audio Deepfake Explanations

Related

• Self-supervised learning with diffusion-based multichannel speech enhancement for speaker verification under noisy conditions
• Extended U-Net for Speaker Verification in Noisy Environments
• Pushing the Frontiers of Self-Distillation Prototypes Network with Dimension Regularization and Score Normalization
• A Study on Speech Enhancement Based on Diffusion Probabilistic Model
• Noise-aware Speech Enhancement using Diffusion Probabilistic Model