Unsupervised Multi-channel Speech Dereverberation via Diffusion

2025/08/04 by Yulun Wu, Wu, Yulun, Zhongweiyang Xu +6
Computer Science · Neuroscience · #Anechoic chamber #Constraint (computer-aided design) #Diffusion #Hearing Loss and Rehabilitation #Impulse (physics) #Impulse response #Microphone #Reverberation #Sampling (signal processing) #Speech Recognition and Synthesis #Speech and Audio Processing #Speech enhancement

paper · open access · doi:10.48550/arxiv.2508.02071

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2025/08/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We consider the problem of multi-channel single-speaker blind dereverberation, where multi-channel mixtures are used to recover the clean anechoic speech. To solve this problem, we propose USD-DPS, Unsupervised Speech Dereverberation via Diffusion Posterior Sampling. USD-DPS uses an unconditional clean speech diffusion model as a strong prior to solve the problem by posterior sampling. At each diffusion sampling step, we estimate all microphone channels' room impulse responses (RIRs), which are further used to enforce a multi-channel mixture consistency constraint for diffusion guidance. For multi-channel RIR estimation, we estimate reference-channel RIR by optimizing RIR parameters of a sub-band RIR signal model, with the Adam optimizer. We estimate non-reference channels' RIRs analytically using forward convolutive prediction (FCP). We found that this combination provides a good balance between sampling efficiency and RIR prior modeling, which shows superior performance among unsupervised dereverberation approaches. An audio demo page is provided in https://usddps.github.io/USDDPS_demo/.

Citations

• ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior
• Unsupervised Estimation of Nonlinear Audio Effects: Comparing Diffusion-Based and Adversarial approaches
• HRTF Estimation using a Score-based Prior
• Unsupervised Blind Joint Dereverberation and Room Acoustics Estimation with Diffusion Models
• USDnet: Unsupervised Speech Dereverberation via Neural Forward Filtering
• Zero-Shot Duet Singing Voices Separation with Diffusion Models
• Fast Diffusion EM: a diffusion model for blind inverse problems with application to deconvolution
• Diffusion Posterior Sampling for Informed Single-Channel Dereverberation
• Blind Audio Bandwidth Extension: A Diffusion-Based Zero-Shot Approach
• UNSSOR: Unsupervised Neural Speech Separation by Leveraging Over-determined Training Mixtures
• Moûsai: Text-to-Music Generation with Long-Context Latent Diffusion
• TF-GridNet: Integrating Full- and Sub-Band Modeling for Speech Separation
• Parallel Diffusion Models of Operator and Image for Blind Inverse Problems
• Unsupervised vocal dereverberation with diffusion-based generative models
• Solving Audio Inverse Problems with a Diffusion Model
• Diffusion Posterior Sampling for General Noisy Inverse Problems
• Elucidating the Design Space of Diffusion-Based Generative Models
• Multichannel Speech Separation with Narrow-band Conformer
• Multi-channel Narrow-band Deep Speech Separation with Full-band Permutation Invariant Training
• Convolutive Prediction for Monaural Speech Dereverberation and Noisy-Reverberant Speaker Separation
• Convolutive Prediction for Reverberant Speech Separation
• Score-Based Generative Modeling through Stochastic Differential Equations
• Denoising Diffusion Probabilistic Models
• Multi-Microphone Complex Spectral Mapping for Speech Dereverberation
• LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech
• SDR - half-baked or well done?

Related

• BUDDy: Single-Channel Blind Unsupervised Dereverberation with Diffusion Models
• Diffusion Posterior Sampling for Informed Single-Channel Dereverberation
• Unsupervised Blind Joint Dereverberation and Room Acoustics Estimation with Diffusion Models
• USDnet: Unsupervised Speech Dereverberation via Neural Forward Filtering
• ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior