Post-Training Quantization for Audio Diffusion Transformers

2025/09/30 by Tanmay Khandelwal, Magdalena Fuentes, Khandelwal, Tanmay +1 · 1 citation
Computer Science · #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Signal Denoising Methods #Neural Networks and Applications #Sound (cs.SD) #Speech and Audio Processing #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2510.00313

openalex publication_date 2025/09/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Diffusion Transformers (DiTs) enable high-quality audio synthesis but are often computationally intensive and require substantial storage, which limits their practical deployment. In this paper, we present a comprehensive evaluation of post-training quantization (PTQ) techniques for audio DiTs, analyzing the trade-offs between static and dynamic quantization schemes. We explore two practical extensions (1) a denoising-timestep-aware smoothing method that adapts quantization scales per-input-channel and timestep to mitigate activation outliers, and (2) a lightweight low-rank adapter (LoRA)-based branch derived from singular value decomposition (SVD) to compensate for residual weight errors. Using Stable Audio Open we benchmark W8A8 and W4A8 configurations across objective metrics and human perceptual ratings. Our results show that dynamic quantization preserves fidelity even at lower precision, while static methods remain competitive with lower latency. Overall, our findings show that low-precision DiTs can retain high-fidelity generation while reducing memory usage by up to 79%.

Citations

• AudioX: A Unified Framework for Anything-to-Audio Generation
• Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Grouping
• Sketch2Sound: Controllable Audio Generation via Time-Varying Signals and Sonic Imitations
• SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models
• ImmerseDiffusion: A Generative Spatial Audio Latent Diffusion Model
• Token Caching for Diffusion Transformer Acceleration
• PTQ4ADM: Post-Training Quantization for Efficient Text Conditional Audio Diffusion Models
• DiTAS: Quantizing Diffusion Transformers via Enhanced Activation Smoothing
• Stable Audio Open
• DEX-TTS: Diffusion-based EXpressive Text-to-Speech with Style Modeling on Time Variability
• PTQ4DiT: Post-training Quantization for Diffusion Transformers
• Fast Timing-Conditioned Latent Audio Diffusion
• DeepCache: Accelerating Diffusion Models for Free
• EDMSound: Spectrogram Based Diffusion Models for Efficient and High-Quality Audio Synthesis
• Controllable Music Production with Diffusion Models and Guidance Gradients
• Towards Accurate Post-training Quantization for Diffusion Models
• PTQD: Accurate Post-Training Quantization for Diffusion Models
• Noise2Music: Text-conditioned Music Generation with Diffusion Models
• Q-Diffusion: Quantizing Diffusion Models
• AudioLDM: Text-to-Audio Generation with Latent Diffusion Models
• Scalable Diffusion Models with Transformers
• Post-training Quantization on Diffusion Models
• SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
• All are Worth Words: A ViT Backbone for Diffusion Models
• Post-training Quantization for Neural Networks with Provable Guarantees
• LoRA: Low-Rank Adaptation of Large Language Models
• Attention Is All You Need

Cited by

• Quantizing Whisper-small: How design choices affect ASR performance

Related

• PTQ4ADM: Post-Training Quantization for Efficient Text Conditional Audio Diffusion Models
• Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers
• DiTAS: Quantizing Diffusion Transformers via Enhanced Activation Smoothing
• PTQ4DiT: Post-training Quantization for Diffusion Transformers
• DVD-Quant: Data-free Video Diffusion Transformers Quantization