2023/05/22 by Yu-Shan Tai, Tai, Yu-Shan, Ming-Guang Lin +3
Engineering · #Advanced Memory and Neural Computing #CCD and CMOS Imaging Sensors #FOS: Electrical engineering #Image and Video Processing (eess.IV) #Infrared Target Detection Methodologies #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2305.12901
openalex publication_date 2023/05/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Vision transformers (ViTs) have achieved remarkable performance in various computer vision tasks. However, intensive memory and computation requirements impede ViTs from running on resource-constrained edge devices. Due to the non-normally distributed values after Softmax and GeLU, post-training quantization on ViTs results in severe accuracy degradation. Moreover, conventional methods fail to address the high channel-wise variance in LayerNorm. To reduce the quantization loss and improve classification accuracy, we propose a two-scaled post-training quantization scheme for vision transformer (TSPTQ-ViT). We design the value-aware two-scaled scaling factors (V-2SF) specialized for post-Softmax and post-GeLU values, which leverage the bit sparsity in non-normal distribution to save bit-widths. In addition, the outlier-aware two-scaled scaling factors (O-2SF) are introduced to LayerNorm, alleviating the dominant impacts from outlier values. Our experimental results show that the proposed methods reach near-lossless accuracy drops (<0.5%) on the ImageNet classification task under 8-bit fully quantized ViTs.