BitTTS: Highly Compact Text-to-Speech Using 1.58-bit Quantization and Weight Indexing

2025/06/04 by Masaya Kawamura, Kawamura, Masaya, Takuya Hasumi +5
Computer Science · #Advanced Data Compression Techniques #Audio and Speech Processing (eess.AS) #Embedded Systems Design Techniques #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Signal Processing (eess.SP) #Sound (cs.SD) #Speech Recognition and Synthesis #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2506.03515

openalex publication_date 2025/06/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper proposes a highly compact, lightweight text-to-speech (TTS) model for on-device applications. To reduce the model size, the proposed model introduces two techniques. First, we introduce quantization-aware training (QAT), which quantizes model parameters during training to as low as 1.58-bit. In this case, most of 32-bit model parameters are quantized to ternary values -1, 0, 1. Second, we propose a method named weight indexing. In this method, we save a group of 1.58-bit weights as a single int8 index. This allows for efficient storage of model parameters, even on hardware that treats values in units of 8-bit. Experimental results demonstrate that the proposed method achieved 83 % reduction in model size, while outperforming the baseline of similar model size without quantization in synthesis quality.

Citations

• The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
• BitNet: Scaling 1-bit Transformers for Large Language Models
• LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpus
• 2-bit Conformer quantization for automatic speech recognition
• A Review of Deep Learning Techniques for Speech Processing
• Lightweight and High-Fidelity End-to-End Text-to-Speech with Multi-Band Generation and Inverse Short-Time Fourier Transform
• Basic Binary Convolution Unit for Binarized Image Restoration Network
• Sub-8-Bit Quantization Aware Training for 8-Bit Neural Network Accelerator with On-Device Speech Recognition
• JETS: Jointly Training FastSpeech2 and HiFi-GAN for End to End Text to Speech
• Nix-TTS: Lightweight and End-to-End Text-to-Speech via Module-wise Distillation
• 4-bit Conformer with Native Quantization Aware Training for Speech Recognition
• A Survey on Neural Speech Synthesis
• A White Paper on Neural Network Quantization
• A Survey of Quantization Methods for Efficient Neural Network Inference
• LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Search
• HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
• SpeedySpeech: Efficient Neural Speech Synthesis
• ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkit
• An Exponential Learning Rate Schedule for Deep Learning
• XNOR-Net++: Improved Binary Neural Networks
• Neural Architecture Optimization
• ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech
• Quantization and Training of Neural Networks for Efficient\n Integer-Arithmetic-Only Inference
• Learning Transferable Architectures for Scalable Image Recognition
• Attention Is All You Need
• Layer Normalization
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural\n Networks
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation

Related

• Efficient Speech Representation Learning with Low-Bit Quantization
• Optimizing Large Language Models through Quantization: A Comparative Analysis of PTQ and QAT Techniques
• IndexTTS: An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech System
• Squat: Quant Small Language Models on the Edge
• PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models