EfficientQuant: An Efficient Post-Training Quantization for CNN-Transformer Hybrid Models on Edge Devices

2025/06/05 by Saha, Shaibal, Xu, Lanyu · 1 citation
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2506.11093

Abstract

Hybrid models that combine convolutional and transformer blocks offer strong performance in computer vision (CV) tasks but are resource-intensive for edge deployment. Although post-training quantization (PTQ) can help reduce resource demand, its application to hybrid models remains limited. We propose EfficientQuant, a novel structure-aware PTQ approach that applies uniform quantization to convolutional blocks and log₂ quantization to transformer blocks. EfficientQuant achieves 2.5 × - 8.7 × latency reduction with minimal accuracy loss on the ImageNet-1K dataset. It further demonstrates low latency and memory efficiency on edge devices, making it practical for real-world deployment.

Citations

• Vision Transformers on the Edge: A Comprehensive Survey of Model Compression and Acceleration Strategies
• AdaLog: Post-Training Quantization for Vision Transformers with Adaptive Logarithm Quantizer
• Q-HyViT: Post-Training Quantization of Hybrid Vision Transformers with Bridge Block Reconstruction for IoT Systems
• RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformers
• Rethinking Vision Transformers for MobileNet Size and Speed
• EdgeNeXt: Efficiently Amalgamated CNN-Transformer Architecture for Mobile Vision Applications
• Separable Self-attention for Mobile Vision Transformers
• EfficientFormer: Vision Transformers at MobileNet Speed
• BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers
• PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantization
• MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer
• HPTQ: Hardware-Friendly Post Training Quantization
• Intriguing Properties of Vision Transformers
• A Survey of Quantization Methods for Efficient Neural Network Inference
• UNETR: Transformers for 3D Medical Image Segmentation
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• EasyQuant: Post-training Quantization via Scale Optimization
• Once-for-All: Train One Network and Specialize it for Efficient Deployment
• Low-bit Quantization of Neural Networks for Efficient Inference
• Improving Neural Network Quantization without Retraining using Outlier Channel Splitting
• Quantization for Rapid Deployment of Deep Neural Networks
• Learning to Quantize Deep Networks by Optimizing Quantization Intervals\n with Task Loss
• PACT: Parameterized Clipping Activation for Quantized Neural Networks
• Quantization and Training of Neural Networks for Efficient\n Integer-Arithmetic-Only Inference

Cited by

• Same Predictions, Different Reasons: The Effect of Quantization on Model Explanations

Related

• Q-HyViT: Post-Training Quantization of Hybrid Vision Transformers with Bridge Block Reconstruction for IoT Systems
• PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantization
• FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer
• Post-Training Quantization for Vision Transformer
• MPTQ-ViT: Mixed-Precision Post-Training Quantization for Vision Transformer