Accelerating Depthwise Separable Convolutions on Ultra-Low-Power Devices

2024/06/18 by Francesco Daghero, Daghero, Francesco, Alessio Burrello +7
Computer Science · Engineering · Physics and Astronomy · #Advanced MEMS and NEMS Technologies #Digital Filter Design and Implementation #Distributed #Electromagnetic Scattering and Analysis #FOS: Computer and information sciences #Machine Learning (cs.LG) #Parallel #and Cluster Computing (cs.DC)

paper · pdf · doi:10.48550/arxiv.2406.12478

openalex publication_date 2024/06/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Depthwise separable convolutions are a fundamental component in efficient Deep Neural Networks, as they reduce the number of parameters and operations compared to traditional convolutions while maintaining comparable accuracy. However, their low data reuse opportunities make deploying them notoriously difficult. In this work, we perform an extensive exploration of alternatives to fuse the depthwise and pointwise kernels that constitute the separable convolutional block. Our approach aims to minimize time-consuming memory transfers by combining different data layouts. When targeting a commercial ultra-low-power device with a three-level memory hierarchy, the GreenWaves GAP8 SoC, we reduce the latency of end-to-end network execution by up to 11.40%. Furthermore, our kernels reduce activation data movements between L2 and L1 memories by up to 52.97%.

Related

• DeepDive: An Integrative Algorithm/Architecture Co-Design for Deep Separable Convolutional Neural Networks
• USEFUSE: Uniform Stride for Enhanced Performance in Fused Layer Architecture of Deep Neural Networks
• High Performance Zero-Memory Overhead Direct Convolutions
• Fusing Depthwise and Pointwise Convolutions for Efficient Inference on GPUs
• FuSeConv: Fully Separable Convolutions for Fast Inference on Systolic Arrays