Neural Network Distiller: A Python Package For DNN Compression Research

2019/10/27 by Neta Zmora, Guy Jacob, Zmora, Neta +7
Computer Science · Medicine · #Advanced Neural Network Applications #COVID-19 diagnosis using AI #Computational Physics and Python Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.1910.12232

openalex publication_date 2019/10/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper presents the philosophy, design and feature-set of Neural Network Distiller, an open-source Python package for DNN compression research. Distiller is a library of DNN compression algorithms implementations, with tools, tutorials and sample applications for various learning tasks. Its target users are both engineers and researchers, and the rich content is complemented by a design-for-extensibility to facilitate new research. Distiller is open-source and is available on Github at https://github.com/NervanaSystems/distiller.

Citations

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Distilling the Knowledge in a Neural Network
• MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Exploring Sparsity in Recurrent Neural Networks
• Pruning Convolutional Neural Networks for Resource Efficient Inference
• Attention-Based Guided Structured Sparsity of Deep Neural Networks
• SinReQ: Generalized Sinusoidal Regularization for Low-Bitwidth Deep\n Quantized Training

Related

• DP-Net: Dynamic Programming Guided Deep Neural Network Compression
• Distilling Neural Networks for Greener and Faster Dependency Parsing
• Evaluation Metrics for DNNs Compression
• Weight Distillation: Transferring the Knowledge in Neural Network Parameters
• A Survey on Deep Neural Network Compression: Challenges, Overview, and Solutions