Active Speech Enhancement: Active Speech Denoising Decliping and Deveraberation

2025/05/22 by Ofir Yaish, Yaish, Ofir, Yehuda Mishaly +3
Computer Science · Engineering · Neuroscience · #Advanced Adaptive Filtering Techniques #Artificial Intelligence (cs.AI) #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Hearing Loss and Rehabilitation #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2505.16911

openalex publication_date 2025/05/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce a new paradigm for active sound modification: Active Speech Enhancement (ASE). While Active Noise Cancellation (ANC) algorithms focus on suppressing external interference, ASE goes further by actively shaping the speech signal -- both attenuating unwanted noise components and amplifying speech-relevant frequencies -- to improve intelligibility and perceptual quality. To enable this, we propose a novel Transformer-Mamba-based architecture, along with a task-specific loss function designed to jointly optimize interference suppression and signal enrichment. Our method outperforms existing baselines across multiple speech processing tasks -- including denoising, dereverberation, and declipping -- demonstrating the effectiveness of active, targeted modulation in challenging acoustic environments.

Citations

• DiTSE: High-Fidelity Generative Speech Enhancement via Latent Diffusion Transformers
• Deep Active Speech Cancellation with Mamba-Masking Network
• GAN-Based Speech Enhancement for Low SNR Using Latent Feature Conditioning
• Wave-U-Mamba: An End-To-End Framework For High-Quality And Efficient Speech Super Resolution
• Rethinking Mamba in Speech Processing by Self-Supervised Models
• TF-Mamba: A Time-Frequency Network for Sound Source Localization
• SELD-Mamba: Selective State-Space Model for Sound Event Localization and Detection with Source Distance Estimation
• Speech Slytherin: Examining the Performance and Efficiency of Mamba for Speech Separation, Recognition, and Synthesis
• Open-Source Conversational AI with SpeechBrain 1.0
• An Exploration of Length Generalization in Transformer-Based Speech Enhancement
• RawBMamba: End-to-End Bidirectional State Space Model for Audio Deepfake Detection
• Audio Mamba: Bidirectional State Space Model for Audio Representation Learning
• Audio Mamba: Bidirectional State Space Model for Audio Representation Learning
• Audio Mamba: Selective State Spaces for Self-Supervised Audio Representations
• Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
• Audio Mamba: Pretrained Audio State Space Model For Audio Tagging
• Mamba in Speech: Towards an Alternative to Self-Attention
• An Investigation of Incorporating Mamba for Speech Enhancement
• Jamba: A Hybrid Transformer-Mamba Language Model
• Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separation
• Multichannel Long-Term Streaming Neural Speech Enhancement for Static and Moving Speakers
• Mamba: Linear-Time Sequence Modeling with Selective State Spaces
• Diffusion-based speech enhancement with a weighted generative-supervised learning loss
• Cross-Attention is all you need: Real-Time Streaming Transformers for Personalised Speech Enhancement
• High Fidelity Neural Audio Compression
• A Hybrid SFANC-FxNLMS Algorithm for Active Noise Control Based on Deep Learning
• Speech Enhancement and Dereverberation with Diffusion-based Generative Models
• Speech Enhancement with Score-Based Generative Models in the Complex STFT Domain
• Conditional Diffusion Probabilistic Model for Speech Enhancement
• SoundStream: An End-to-End Neural Audio Codec
• Nonlinear Acoustic Echo Cancellation with Deep Learning
• MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement
• TSTNN: Two-stage Transformer based Neural Network for Speech Enhancement in the Time Domain
• Attention is All You Need in Speech Separation
• DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement
• Real Time Speech Enhancement in the Waveform Domain
• Conformer: Convolution-augmented Transformer for Speech Recognition
• Conformer: Convolution-augmented Transformer for Speech Recognition
• Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation
• MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement
• A Wavenet for Speech Denoising
• Attention Is All You Need
• SEGAN: Speech Enhancement Generative Adversarial Network
• Adam: A Method for Stochastic Optimization
• A Fast Learning Algorithm for Deep Belief Nets
• Active noise control: a tutorial review
• Assessment for automatic speech recognition: II. NOISEX-92: A database and an experiment to study the effect of additive noise on speech recognition systems
• Suppression of acoustic noise in speech using spectral subtraction
• All-pole modeling of degraded speech
• Image method for efficiently simulating small-room acoustics
• SPMamba: State-space model is all you need in speech separation

Related

• Deep Active Speech Cancellation with Mamba-Masking Network
• Active acoustic enhancement systems: A review
• EDNet: A Versatile Speech Enhancement Framework with Gating Mamba Mechanism and Phase Shift-Invariant Training
• VC-ENHANCE: Speech Restoration with Integrated Noise Suppression and Voice Conversion
• Restorative Speech Enhancement: A Progressive Approach Using SE and Codec Modules