2020/11/05 by Khaled Koutini, Koutini, Khaled, Florian Henkel +5
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Music and Audio Processing #Neural and Evolutionary Computing (cs.NE) #Sound (cs.SD) #Speech Recognition and Synthesis #Speech and Audio Processing
paper · pdf · doi:10.48550/arxiv.2011.02955
openalex publication_date 2020/11/05 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28
Deep Neural Networks are known to be very demanding in terms of computing and\nmemory requirements. Due to the ever increasing use of embedded systems and\nmobile devices with a limited resource budget, designing low-complexity models\nwithout sacrificing too much of their predictive performance gained great\nimportance. In this work, we investigate and compare several well-known methods\nto reduce the number of parameters in neural networks. We further put these\ninto the context of a recent study on the effect of the Receptive Field (RF) on\na model's performance, and empirically show that we can achieve high-performing\nlow-complexity models by applying specific restrictions on the RFs, in\ncombination with parameter reduction methods. Additionally, we propose a\nfilter-damping technique for regularizing the RF of models, without altering\ntheir architecture and changing their parameter counts. We will show that\nincorporating this technique improves the performance in various low-complexity\nsettings such as pruning and decomposed convolution. Using our proposed filter\ndamping, we achieved the 1st rank at the DCASE-2020 Challenge in the task of\nLow-Complexity Acoustic Scene Classification.\n