NeuroTrainer: An Intelligent Memory Module for Deep Learning Training

2017/10/12 by Duck-Hwan Kim, Kim, Duckhwan, Taesik Na +5
Computer Science · Engineering · #Advanced Memory and Neural Computing #Advanced Neural Network Applications #FOS: Computer and information sciences #Ferroelectric and Negative Capacitance Devices #Hardware Architecture (cs.AR)

paper · pdf · doi:10.48550/arxiv.1710.04347

openalex publication_date 2017/10/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper presents, NeuroTrainer, an intelligent memory module with in-memory accelerators that forms the building block of a scalable architecture for energy efficient training for deep neural networks. The proposed architecture is based on integration of a homogeneous computing substrate composed of multiple processing engines in the logic layer of a 3D memory module. NeuroTrainer utilizes a programmable data flow based execution model to optimize memory mapping and data re-use during different phases of training operation. A programming model and supporting architecture utilizes the flexible data flow to efficiently accelerate training of various types of DNNs. The cycle level simulation and synthesized design in 15nm FinFET showspower efficiency of 500 GFLOPS/W, and almost similar throughput for a wide range of DNNs including convolutional, recurrent, multi-layer-perceptron, and mixed (CNN+RNN) networks

Citations

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Learning with Limited Numerical Precision
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling

Related

• Survey on Large Scale Neural Network Training
• A Scalable Near-Memory Architecture for Training Deep Neural Networks on Large In-Memory Datasets
• Hybrid In-memory Computing Architecture for the Training of Deep Neural Networks
• Reducing Data Motion to Accelerate the Training of Deep Neural Networks
• Acceleration of Deep Neural Network Training with Resistive Cross-Point Devices: Design Considerations