CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs

2018/01/19 by Lai, Liangzhen, Suda, Naveen, Chandra, Vikas · 9 citations
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Mathematical Software (cs.MS) #Neural and Evolutionary Computing (cs.NE)

paper · doi:10.48550/arxiv.1801.06601

Abstract

Deep Neural Networks are becoming increasingly popular in always-on IoT edge devices performing data analytics right at the source, reducing latency as well as energy consumption for data communication. This paper presents CMSIS-NN, efficient kernels developed to maximize the performance and minimize the memory footprint of neural network (NN) applications on Arm Cortex-M processors targeted for intelligent IoT edge devices. Neural network inference based on CMSIS-NN kernels achieves 4.6X improvement in runtime/throughput and 4.9X improvement in energy efficiency.

Cited by

• The SpiNNaker2 chip: a many-core platform for flexible and scalable brain-inspired computing
• Semi-Supervised Online Learning on the Edge by Transforming Knowledge from Teacher Models
• SynapticOS: An Inference-First Runtime Architecture for Neural Processing Units on Resource-Constrained Microcontrollers
• Enabling Multi-Species Bird Classification on Low-Power Bioacoustic Loggers
• MaRVIn: A Cross-Layer Mixed-Precision RISC-V Framework for DNN Inference, from ISA Extension to Hardware Acceleration
• Decentor-V: Lightweight ML Training on Low-Power RISC-V Edge Devices
• Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications
• SNAP-UQ: Self-supervised Next-Activation Prediction for Single-Pass Uncertainty in TinyML
• MiCo: End-to-End Mixed Precision Neural Network Co-Exploration Framework for Edge AI

Related

• FANN-on-MCU: An Open-Source Toolkit for Energy-Efficient Neural Network Inference at the Edge of the Internet of Things
• DNN is not all you need: Parallelizing Non-Neural ML Algorithms on Ultra-Low-Power IoT Processors
• Efficient Neural Network Deployment for Microcontroller
• Architectural Implications of Neural Network Inference for High Data-Rate, Low-Latency Scientific Applications
• ONNX-to-Hardware Design Flow for Adaptive Neural-Network Inference on FPGAs