PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models

2025/09/21 by He Xiao, Xiao, He, Yang, Runming +13
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2509.16989

openalex publication_date 2025/09/21 · openalex created_date 2025/10/16 · openalex updated_date 2026/07/28

Abstract

Post-training quantization (PTQ) of large language models (LLMs) to extremely low bit-widths remains challenging due to the fundamental trade-off between computational efficiency and representational capacity. While existing ultra-low-bit methods rely on binary approximations or quantization-aware training(QAT), they often suffer from either limited representational capacity or huge training resource overhead. We introduce PTQ to Trit-Planes (PTQTP), a structured PTQ framework that decomposes weight matrices into dual ternary -1, 0, 1 trit-planes. This approach achieves multiplication-free additive inference by decoupling weights into discrete topology (trit-planes) and continuous magnitude (scales), effectively enabling high-fidelity sparse approximation. PTQTP provides: (1) a theoretically grounded progressive approximation algorithm ensuring global weight consistency; (2) model-agnostic deployment without architectural modifications; and (3) uniform ternary operations that eliminate mixed-precision overhead. Comprehensive experiments on LLaMA3.x and Qwen3 (0.6B-70B) demonstrate that PTQTP significantly outperforms sub-4bit PTQ methods on both language reasoning tasks and mathematical reasoning as well as coding. PTQTP rivals the 1.58-bit QAT performance while requiring only single-hour quantization compared to 10-14 GPU days for training-based methods, and the end-to-end inference speed achieves 4.63× faster than the FP16 baseline model, establishing a new and practical solution for efficient LLM deployment in resource-constrained environments. Code will available at https://github.com/HeXiao-55/PTQTP.

Citations

• BitNet b1.58 2B4T Technical Report
• SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
• DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
• Qwen2.5 Technical Report
• ARB-LLM: Alternating Refined Binarizations for Large Language Models
• The Llama 3 Herd of Models
• QTIP: Quantization with Trellises and Incoherence Processing
• MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
• QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
• The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
• QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks
• BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
• Extreme Compression of Large Language Models via Additive Quantization
• FlightLLM: Efficient Large Language Model Inference with a Complete Mapping Flow on FPGAs
• OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models
• QuIP: 2-Bit Quantization of Large Language Models With Guarantees
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
• Context-Based Trit-Plane Coding for Progressive Image Compression
• LLaMA: Open and Efficient Foundation Language Models
• DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generation
• LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models
• DPICT: Deep Progressive Image Compression Using Trit-Planes
• Program Synthesis with Large Language Models
• Evaluating Large Language Models Trained on Code
• Carbon Emissions and Large Neural Network Training
• Measuring Massive Multitask Language Understanding
• Language Models are Few-Shot Learners
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• PIQA: Reasoning about Physical Commonsense in Natural Language
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
• HellaSwag: Can a Machine Really Finish Your Sentence?
• Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
• Pointer Sentinel Mixture Models

Cited by

• Exploring Layer-wise Information Effectiveness for Post-Training Quantization in Small Language Models

Related

• ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models
• PT²-LLM: Post-Training Ternarization for Large Language Models
• PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models
• TesseraQ: Ultra Low-Bit LLM Post-Training Quantization with Block Reconstruction
• CBQ: Cross-Block Quantization for Large Language Models