Universal Speech Enhancement with Regression and Generative Mamba

2025/05/27 by Chao, Rong, Rauf Nasretdinov, Nasretdinov, Rauf +8 · 5 citations
Computer Science · Engineering · #Advanced Adaptive Filtering Techniques #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Indoor and Outdoor Localization Technologies #Sound (cs.SD) #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2505.21198

openalex publication_date 2025/05/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The Interspeech 2025 URGENT Challenge aimed to advance universal, robust, and generalizable speech enhancement by unifying speech enhancement tasks across a wide variety of conditions, including seven different distortion types and five languages. We present Universal Speech Enhancement Mamba (USEMamba), a state-space speech enhancement model designed to handle long-range sequence modeling, time-frequency structured processing, and sampling frequency-independent feature extraction. Our approach primarily relies on regression-based modeling, which performs well across most distortions. However, for packet loss and bandwidth extension, where missing content must be inferred, a generative variant of the proposed USEMamba proves more effective. Despite being trained on only a subset of the full training data, USEMamba achieved 2nd place in Track 1 during the blind test phase, demonstrating strong generalization across diverse conditions.

Citations

• AnyEnhance: A Unified Generative Model with Prompt-Guidance and Self-Critic for Voice Enhancement
• Generative Speech Foundation Model Pretraining for High-Quality Speech Extraction and Restoration
• MaskSR: Masked Language Model for Full-band Speech Restoration
• An Investigation of Incorporating Mamba for Speech Enhancement
• SpeechBERTScore: Reference-Aware Automatic Evaluation of Speech Generation Leveraging NLP Evaluation Metrics
• Mamba: Linear-Time Sequence Modeling with Selective State Spaces
• Generative Pre-training for Speech with Flow Matching
• Toward Universal Speech Enhancement for Diverse Input Conditions
• Spiking Structured State Space Model for Monaural Speech Enhancement
• SpeechX: Neural Codec Language Model as a Versatile Speech Transformer
• Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale
• Miipher: A Robust Speech Restoration Model Integrating Self-Supervised Speech and Text Representations
• Scalable Diffusion Models with Transformers
• Flow Matching for Generative Modeling
• On the Parameterization and Initialization of Diagonal State Space Models
• Universal Speech Enhancement with Score-based Diffusion
• UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022
• DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors
• DPT-FSNet: Dual-path Transformer Based Full-band and Sub-band Fusion Network for Speech Enhancement
• HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks
• SDR - half-baked or well done?
• Supervised Speech Separation Based on Deep Learning: An Overview

Cited by

• TF-Restormer: Complex Spectral Prediction for Speech Restoration
• An overview of neural architectures for self-supervised audio representation learning from masked spectrograms
• End-to-End Audio-Visual Learning for Cochlear Implant Sound Coding in Noisy Environments
• Leveraging Mamba with Full-Face Vision for Audio-Visual Speech Enhancement
• EDNet: A Versatile Speech Enhancement Framework with Gating Mamba Mechanism and Phase Shift-Invariant Training

Related

• An Investigation of Incorporating Mamba for Speech Enhancement
• Improving Speech Enhancement by Cross- and Sub-band Processing with State Space Model
• MambAttention: Mamba with Multi-Head Attention for Generalizable Single-Channel Speech Enhancement
• Mamba-SEUNet: Mamba UNet for Monaural Speech Enhancement
• Selective State Space Model for Monaural Speech Enhancement