CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs

2018/01/19 by Liangzhen Lai, Lai, Liangzhen, Naveen Suda +3 · 17 citations
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Mathematical Software (cs.MS) #Neural and Evolutionary Computing (cs.NE) #cs.LG #cs.MS #cs.NE

paper · pdf · doi:10.48550/arxiv.1801.06601

arxiv created 2018/01/19 · arxiv updated 2018/01/23

Abstract

Deep Neural Networks are becoming increasingly popular in always-on IoT edge devices performing data analytics right at the source, reducing latency as well as energy consumption for data communication. This paper presents CMSIS-NN, efficient kernels developed to maximize the performance and minimize the memory footprint of neural network (NN) applications on Arm Cortex-M processors targeted for intelligent IoT edge devices. Neural network inference based on CMSIS-NN kernels achieves 4.6X improvement in runtime/throughput and 4.9X improvement in energy efficiency.

Cited by

• The SpiNNaker2 chip: a many-core platform for flexible and scalable brain-inspired computing
• Semi-Supervised Online Learning on the Edge by Transforming Knowledge from Teacher Models
• SynapticOS: An Inference-First Runtime Architecture for Neural Processing Units on Resource-Constrained Microcontrollers
• Enabling Multi-Species Bird Classification on Low-Power Bioacoustic Loggers
• MaRVIn: A Cross-Layer Mixed-Precision RISC-V Framework for DNN Inference, from ISA Extension to Hardware Acceleration
• Decentor-V: Lightweight ML Training on Low-Power RISC-V Edge Devices
• Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications
• SNAP-UQ: Self-supervised Next-Activation Prediction for Single-Pass Uncertainty in TinyML
• MiCo: End-to-End Mixed Precision Neural Network Co-Exploration Framework for Edge AI
• From Tiny Machine Learning to Tiny Deep Learning: A Survey
• MUDAS: Mote-scale Unsupervised Domain Adaptation in Multi-label Sound Classification
• msf-CNN: Patch-based Multi-Stage Fusion with Convolutional Neural Networks for TinyML
• A probabilistic framework for dynamic quantization
• Cross-Core Inference Offload as an Operating-System Service on Dual-Core Microcontrollers
• Inference Pipelines as Operating-System Objects: Priority Scheduling and Constant-Footprint Streaming for Microcontroller Neural Inference
• Neuromorphic Parameter Estimation for Power Converter Health Monitoring Using Spiking Neural Networks
• Diffusion Models on the Edge: Challenges, Optimizations, and Applications

Related

• FANN-on-MCU: An Open-Source Toolkit for Energy-Efficient Neural Network Inference at the Edge of the Internet of Things
• DNN is not all you need: Parallelizing Non-Neural ML Algorithms on Ultra-Low-Power IoT Processors
• Efficient Neural Network Deployment for Microcontroller
• Architectural Implications of Neural Network Inference for High Data-Rate, Low-Latency Scientific Applications
• ONNX-to-Hardware Design Flow for Adaptive Neural-Network Inference on FPGAs