CAT: Compression-Aware Training for bandwidth reduction

2019/09/25 by Chaim Baskin, Baskin, Chaim, Brian Chmiel +9 · 2 citations
Computer Science · #Advanced Data Compression Techniques #Algorithms and Data Compression #Computer Vision and Pattern Recognition (cs.CV) #Embedded Systems Design Techniques #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.1909.11481

openalex publication_date 2019/09/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Convolutional neural networks (CNNs) have become the dominant neural network architecture for solving visual processing tasks. One of the major obstacles hindering the ubiquitous use of CNNs for inference is their relatively high memory bandwidth requirements, which can be a main energy consumer and throughput bottleneck in hardware accelerators. Accordingly, an efficient feature map compression method can result in substantial performance gains. Inspired by quantization-aware training approaches, we propose a compression-aware training (CAT) method that involves training the model in a way that allows better compression of feature maps during inference. Our method trains the model to achieve low-entropy feature maps, which enables efficient compression at inference time using classical transform coding methods. CAT significantly improves the state-of-the-art results reported for quantization. For example, on ResNet-34 we achieve 73.1% accuracy (0.2% degradation from the baseline) with an average representation of only 1.79 bits per value. Reference implementation accompanies the paper at https://github.com/CAT-teams/CAT

Citations

• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
• The State of Sparsity in Deep Neural Networks
• LUTNet: Rethinking Inference in FPGA Soft Logic

Cited by

• KV Cache Transform Coding for Compact Storage in LLM Inference
• Hybrid and Non-Uniform quantization methods using retro synthesis data\n for efficient inference

Related

• Transform-Based Feature Map Compression for CNN Inference
• Compact CNN Structure Learning by Knowledge Distillation
• Efficient Neural Compression with Inference-time Decoding
• And the Bit Goes Down: Revisiting the Quantization of Neural Networks
• Accelerating Convolutional Neural Networks via Activation Map\n Compression