2023/12/16 by Mingbin Xu, Xu, Mingbin, Alex Jin +20 · 2 voices
Computer Science · #Advanced Data Compression Techniques #FOS: Computer and information sciences #Machine Learning (cs.LG) #Performance (cs.PF) #Speech Recognition and Synthesis #Speech and Audio Processing #cs.LG #cs.PF
paper · pdf · doi:10.48550/arxiv.2312.10359
openalex publication_date 2023/12/16 · arxiv published 2023/12/16 · arxiv updated 2024/05/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
With increasingly more powerful compute capabilities and resources in today's devices, traditionally compute-intensive automatic speech recognition (ASR) has been moving from the cloud to devices to better protect user privacy. However, it is still challenging to implement on-device ASR on resource-constrained devices, such as smartphones, smart wearables, and other smart home automation devices. In this paper, we propose a series of model architecture adaptions, neural network graph transformations, and numerical optimizations to fit an advanced Conformer based end-to-end streaming ASR system on resource-constrained devices without accuracy degradation. We achieve over 5.26 times faster than realtime (0.19 RTF) speech recognition on smart wearables while minimizing energy consumption and achieving state-of-the-art accuracy. The proposed methods are widely applicable to other transformer-based server-free AI applications. In addition, we provide a complete theory on optimal pre-normalizers that numerically stabilize layer normalization in any Lp-norm using any floating point precision.