GoQuant: Geometric Orthogonal Residual Projection for Multiplier-Free Power-of-Two Transformer Quantization

2026/05/31 by Maoyang Xiang, Tao Luo, Bo Wang
#cs.LG #cs.AI

paper · pdf

Abstract

The deployment of Large Language Models (LLMs) and Vision Transformers (ViTs) on edge devices is significantly constrained by memory capacity and the critical timing bottlenecks introduced by dense Multiply--Accumulate (MAC) arrays. In the ultra-low-bit regime, logarithmic Power-of-Two (PoT) quantization provides a hardware-efficient alternative by replacing general multiplications in the dominant dot-product computation with bit-shift operations. However, its non-uniform exponential lattice inherently suffers from a Low Angular Resolution Regime, a structural limitation that becomes particularly pronounced below 4-bit precision and can substantially degrade the representation of high-dimensional feature manifolds. To address this geometric limitation, we propose Geometric Orthogonal Residual Projection Quantization (GoQuant), an algorithm--hardware co-design framework for multiplier-reduced low-bit inference. By formulating quantization as a dual-basis geometric projection, GoQuant constructs a higher-resolution residual lattice while retaining a shift-and-add inner-product structure. Its analytical solver further avoids computationally intensive gradient-based or iterative search procedures. The data-free Geometric-Only (GEO) mode quantizes LLaMA-2-7B in only 0.47 minutes, while the Activation-Refined (REF) mode completes full-model quantization in approximately 4.4 minutes.

Citations

• LLMC+: Benchmarking Vision-Language Model Compression with a Plug-and-play Toolkit
• The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithm
• AIQViT: Architecture-Informed Post-Training Quantization for Vision Transformers
• ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization
• SpinQuant: LLM quantization with learned rotations
• QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
• QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks
• BitNet: Scaling 1-bit Transformers for Large Language Models
• QuIP: 2-Bit Quantization of Large Language Models With Guarantees
• AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
• RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformers
• SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
• GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
• QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization
• PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantization
• BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction

Related

• ButterflyQuant: Ultra-low-bit LLM Quantization through Learnable Orthogonal Butterfly Transforms
• P²-ViT: Power-of-Two Post-Training Quantization and Acceleration for Fully Quantized Vision Transformer
• FrameQuant: Flexible Low-Bit Quantization for Transformers
• GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration
• Extremely Low Bit Transformer Quantization for On-Device Neural Machine\n Translation