Towards Real-Time DNN Inference on Mobile Platforms with Model Pruning and Compiler Optimization

2020/04/22 by Wei Niu, Pu Zhao, Niu, Wei +9
Computer Science · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2004.11250

Abstract

High-end mobile platforms rapidly serve as primary computing devices for a wide range of Deep Neural Network (DNN) applications. However, the constrained computation and storage resources on these devices still pose significant challenges for real-time DNN inference executions. To address this problem, we propose a set of hardware-friendly structured model pruning and compiler optimization techniques to accelerate DNN executions on mobile devices. This demo shows that these optimizations can enable real-time mobile execution of multiple DNN applications, including style transfer, DNN coloring and super resolution.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Related

• 26ms Inference Time for ResNet-50: Towards Real-Time Execution of all DNNs on Smartphone
• CPrune: Compiler-Informed Model Pruning for Efficient Target-Aware DNN Execution
• Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration
• Optimizing Multi-DNN Inference on Mobile Devices through Heterogeneous Processor Co-Execution
• GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsity