Benchmarking TPU, GPU, and CPU Platforms for Deep Learning

2019/07/24 by Yu Emma Wang, Gu-Yeon Wei, Wang, Yu Emma +3 · 1 voice · 233 citations
Computer Science · Engineering · Mathematics · #Advanced Neural Network Applications #Artificial intelligence #Benchmark (surveying) #Benchmarking #CUDA #Cloud computing #Computer architecture #Computer science #Convolutional neural network #Deep learning #Ferroelectric and Negative Capacitance Devices #Machine learning #Operating system #Parallel Computing and Optimization Techniques #Parallel computing #Suite #cs.LG #cs.PF #stat.ML

paper · pdf · doi:10.48550/arxiv.1907.10701

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2019/07/24 · arxiv created 2019/10/22 · arxiv updated 2019/10/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Training deep learning models is compute-intensive and there is an industry-wide trend towards hardware specialization to improve performance. To systematically benchmark deep learning platforms, we introduce ParaDnn, a parameterized benchmark suite for deep learning that generates end-to-end models for fully connected (FC), convolutional (CNN), and recurrent (RNN) neural networks. Along with six real-world models, we benchmark Google's Cloud TPU v2/v3, NVIDIA's V100 GPU, and an Intel Skylake CPU platform. We take a deep dive into TPU architecture, reveal its bottlenecks, and highlight valuable lessons learned for future specialized system design. We also provide a thorough comparison of the platforms and find that each has unique strengths for some types of models. Finally, we quantify the rapid performance improvements that specialized software stacks provide for the TPU and GPU platforms.

Citations

• Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
• AI Matrix - Synthetic Benchmarks for DNN
• TVM: An Automated End-to-End Optimizing Compiler for Deep Learning

Cited by

• DSTAR: Accelerating Diffusion Transformers via Spatial and Temporal Redundancy Reduction
• Weight Transformations in Bit-Sliced Crossbar Arrays for Fault Tolerant Computing-in-Memory: Design Techniques and Evaluation Framework
• FLAMMABLE: A Multi-Model Federated Learning Framework with Multi-Model Engagement and Adaptive Batch Sizes
• Characterizing the Performance of Accelerated Jetson Edge Devices for Training Deep Learning Models
• SynergAI: Edge-to-Cloud Synergy for Architecture-Driven High-Performance Orchestration for AI Inference
• Estudio de la eficiencia en la escalabilidad de GPUs para el entrenamiento de Inteligencia Artificial
• Data-Driven Power Modeling and Monitoring via Hardware Performance Counter Tracking
• CMOS+X: Stacking Persistent Embedded Memories based on Oxide Transistors upon GPGPU Platforms
• A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes
• Compact, Large-Scale Photonic Neurons by Modulation-and-Weight Microring Resonators
• Democratizing AI: A Comparative Study in Deep Learning Efficiency and Future Trends in Computational Processing
• Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond
• End-to-End Throughput Benchmarking of Portable Deterministic CNN-Based Signal Processing Pipelines
• Benchmarking the Performance and Energy Efficiency of AI Accelerators for AI Training

Discussions

• Benchmarking TPU, GPU, and CPU Platforms for Deep Learning [hn, 51 points, 7 comments]

Related

• Benchmarking State-of-the-Art Deep Learning Software Tools
• SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems
• Benchmarking the Performance and Energy Efficiency of AI Accelerators for AI Training
• Comparative Analysis of CPU and GPU Profiling for Deep Learning Models
• Hardware Acceleration for Neural Networks: A Comprehensive Survey