On Filter Generalization for Music Bandwidth Extension Using Deep Neural Networks

2020/11/12 by Serkan Sulun, Matthew E. P. Davies · 26 citations
Computer Science · Engineering · Mathematics · Neuroscience · #Acoustic Wave Phenomena Research #Algorithm #Artificial intelligence #Artificial neural network #Audio signal #Bandwidth (computing) #Bandwidth extension #Computer science #Computer vision #Deep learning #Filter (signal processing) #Generalization #Hearing Loss and Rehabilitation #Mathematics #Overfitting #Pattern recognition (psychology) #Speech and Audio Processing #Speech coding #Speech recognition #Telecommunications #cs.AI #cs.LG #cs.SD #eess.AS

paper · pdf · doi:10.1109/jstsp.2020.3037485

published in IEEE Journal of Selected Topics in Signal Processing 15(1), 132-142 (Institute of Electrical and Electronics Engineers) · Qualitative examples on https://serkansulun.com/bwe. Source code on https://github.com/serkansulun/deep-music-enhancer

openalex publication_date 2020/11/12 · arxiv created 2021/01/06 · arxiv updated 2021/01/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

In this paper, we address a subtopic of the broad domain of audio enhancement, namely musical audio bandwidth extension. We formulate the bandwidth extension problem using deep neural networks, where a band-limited signal is provided as input to the network, with the goal of reconstructing a full-bandwidth output. Our main contribution centers on the impact of the choice of low-pass filter when training and subsequently testing the network. For two different state-of-the-art deep architectures, ResNet and U-Net, we demonstrate that when the training and testing filters are matched, improvements in signal-to-noise ratio (SNR) of up to 7 dB can be obtained. However, when these filters differ, the improvement falls considerably and under some training conditions results in a lower SNR than the band-limited input. To circumvent this apparent overfitting to filter shape, we propose a data augmentation strategy which utilizes multiple low-pass filters during training and leads to improved generalization to unseen filtering conditions at test time.

Citations

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Image Super-Resolution Using Deep Convolutional Networks
• An empirical study on evaluation metrics of generative adversarial networks
• Improved Speech Enhancement with the Wave-U-Net
• Bandwidth Extension on Raw Audio via Generative Adversarial Networks
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Investigating Under and Overfitting in Wasserstein Generative Adversarial Networks
• Learning to Denoise Historical Music
• Generative Adversarial Nets