Efficient Processing of Deep Neural Networks: A Tutorial and Survey

2017/03/27 by Vivienne Sze, Sze, Vivienne, Yu-Hsin Chen +6 · 3 voices · 53 citations
Computer Science · Engineering · #Advanced Memory and Neural Computing #Advanced Neural Network Applications #Ferroelectric and Negative Capacitance Devices #cs.CV

paper · pdf · doi:10.48550/arxiv.1703.09039

arxiv published 2017/03/27 · arxiv updated 2017/08/13

Abstract

Deep neural networks (DNNs) are currently widely used for many artificial intelligence (AI) applications including computer vision, speech recognition, and robotics. While DNNs deliver state-of-the-art accuracy on many AI tasks, it comes at the cost of high computational complexity. Accordingly, techniques that enable efficient processing of DNNs to improve energy efficiency and throughput without sacrificing application accuracy or increasing hardware cost are critical to the wide deployment of DNNs in AI systems. This article aims to provide a comprehensive tutorial and survey about the recent advances towards the goal of enabling efficient processing of DNNs. Specifically, it will provide an overview of DNNs, discuss various hardware platforms and architectures that support DNNs, and highlight key trends in reducing the computation cost of DNNs either solely via hardware design changes or via joint hardware design and DNN algorithm changes. It will also summarize various development resources that enable researchers and practitioners to quickly get started in this field, and highlight important benchmarking metrics and design considerations that should be used for evaluating the rapidly growing number of DNN hardware designs, optionally including algorithmic co-designs, being proposed in academia and industry. The reader will take away the following concepts from this article: understand the key design considerations for DNNs; be able to evaluate different DNN hardware implementations with benchmarks and comparison metrics; understand the trade-offs between various hardware architectures and platforms; be able to evaluate the utility of various DNN design techniques for efficient processing; and understand recent implementation trends and opportunities.

Citations

• Gradient-based learning applied to document recognition
• Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
• Do Deep Convolutional Nets Really Need to be Deep and Convolutional?
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Cited by

• Kernel Forge: An Agent Harness for LLM-based Generation and Optimization of CUDA Kernels
• Stacked Intelligent Metasurface-Aided Wave-Domain Signal Processing: From Communications to Sensing and Computing
• An Energy-Efficient Adiabatic Capacitive Neural Network Chip
• Learning Dynamics in Memristor-Based Equilibrium Propagation
• Neural Collapse in Test-Time Adaptation
• RIFT: A Scalable Methodology for LLM Accelerator Fault Assessment using Reinforcement Learning
• SHARe-KAN: Post-Training Vector Quantization for Cache-Resident KAN Inference
• Vision and Causal Learning Based Channel Estimation for THz Communications
• Efficient Kernel Mapping and Comprehensive System Evaluation of LLM Acceleration on a CGLA
• TWEO: Transformers Without Extreme Outliers Enables FP8 Training And Quantization For Dummies
• Physics-Informed Spiking Neural Networks via Conservative Flux Quantization
• Equivariant-Aware Structured Pruning for Efficient Edge Deployment: A Comprehensive Framework with Adaptive Fine-Tuning
• Sex and age determination in European lobsters using AI-Enhanced bioacoustics
• Towards Practical Real-Time Low-Latency Music Source Separation
• SynapticCore-X: A Modular Neural Processing Architecture for Low-Cost FPGA Acceleration
• LILogic Net: Compact Logic Gate Networks with Learnable Connectivity for Efficient Hardware Deployment
• CompressNAS : A Fast and Efficient Technique for Model Compression using Decomposition
• OpenMENA: An Open-Source Memristor Interfacing and Compute Board for Neuromorphic Edge-AI Applications
• Data Trajectory Alignment for LLM Domain Adaptation: A Two-Phase Synthesis Framework for Telecommunications Mathematics
• Synheart Emotion: Privacy-Preserving On-Device Emotion Recognition from Biosignals
• Learning a Decentralized Medium Access Control Protocol for Shared Message Transmission
• A Memory-Efficient Retrieval Architecture for RAG-Enabled Wearable Medical LLMs-Agents
• Lincoln AI Computing Survey (LAICS) and Trends
• TernaryCLIP: Efficiently Compressing Vision-Language Models with Ternary Weights and Distilled Knowledge
• Enhancing Time-Series Anomaly Detection by Integrating Spectral-Residual Bottom-Up Attention with Reservoir Computing
• Multiplier-free In-Memory Vector-Matrix Multiplication Using Distributed Arithmetic
• A Methodology for Transparent Logic-Based Classification Using a Multi-Task Convolutional Tsetlin Machine
• SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions
• GUIDE: Guided Initialization and Distillation of Embeddings
• Benchmarking Deep Learning Convolutions on Energy-constrained CPUs
• High Clockrate Free-space Optical In-Memory Computing
• A Theoretical Framework for Stochastic Activity Prediction in Tensor Accelerator Wallace-Tree Multipliers
• Puzzle: Scheduling Multiple Deep Learning Models on Mobile Device with Heterogeneous Processors
• Evaluating the Energy Efficiency of NPU-Accelerated Machine Learning Inference on Embedded Microcontrollers
• Joint Memory Frequency and Computing Frequency Scaling for Energy-efficient DNN Inference
• Deep Hierarchical Learning with Nested Subspace Networks for Large Language Models
• eIQ Neutron: Redefining Edge-AI Inference with Integrated NPU and Compiler Innovations
• Orthrus: Dual-Loop Automated Framework for System-Technology Co-Optimization
• Deep Learning-based Techniques for Integrated Sensing and Communication Systems: State-of-the-Art, Challenges, and Opportunities
• In-Loop Filtering Using Learned Look-Up Tables for Video Coding
• Breaking SafetyCore: Exploring the Risks of On-Device AI Deployment
• Scenario-based Decision-making Using Game Theory for Interactive Autonomous Driving: A Survey
• Real Time FPGA Based CNNs for Detection, Classification, and Tracking in Autonomous Systems: State of the Art Designs and Optimizations
• Hardwired-Neurons Language Processing Units as General-Purpose Cognitive Substrates
• Application of discrete Ricci curvature in pruning randomly wired neural networks: A case study with chest x-ray classification of COVID-19
• The AI gambit: leveraging artificial intelligence to combat climate change—opportunities, challenges, and recommendations
• Practical GPU Choices for Earth Observation: ResNet-50 Training Throughput on Integrated, Laptop, and Cloud Accelerators
• Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications
• Cross-Layer Design of Vector-Symbolic Computing: Bridging Cognition and Brain-Inspired Hardware Acceleration
• Quantization through Piecewise-Affine Regularization: Optimization and Statistical Guarantees
• Marvel: A Data-centric Compiler for DNN Operators on Spatial Accelerators
• Foundation Models for Demand Forecasting via Dual-Strategy Ensembling
• The Carbon Cost of Conversation, Sustainability in the Age of Language Models

Discussions

• Efficient Processing of Deep Neural Networks: A Tutorial and Survey [hn, 3 points, 0 comments]
• Efficient Processing of Deep Neural Networks: A Tutorial and Survey [hn, 2 points, 0 comments]
• Efficient Processing of Deep Neural Networks [hn, 2 points, 0 comments]

Related

• Efficient Processing of Deep Neural Networks: A Tutorial and Survey
• Learning on Hardware: A Tutorial on Neural Network Accelerators and Co-Processors
• AI Matrix - Synthetic Benchmarks for DNN
• From DNNs to GANs: Review of efficient hardware architectures for deep learning
• Efficient Deep Neural Networks