DVD-Quant: Data-free Video Diffusion Transformers Quantization

2025/05/24 by Zhiteng Li, Li, Zhiteng, Hanxuan Li +14 · 4 citations
Computer Science · #Image and Signal Denoising Methods #Advanced Image Processing Techniques #Computer Graphics and Visualization Techniques

paper · pdf · doi:10.48550/arxiv.2505.18663

Abstract

Diffusion Transformers (DiTs) have emerged as the state-of-the-art architecture for video generation, yet their computational and memory demands hinder practical deployment. While post-training quantization (PTQ) presents a promising approach to accelerate Video DiT models, existing methods suffer from two critical limitations: (1) dependence on computation-heavy and inflexible calibration procedures, and (2) considerable performance deterioration after quantization. To address these challenges, we propose DVD-Quant, a novel Data-free quantization framework for Video DiTs. Our approach integrates three key innovations: (1) Bounded-init Grid Refinement (BGR) and (2) Auto-scaling Rotated Quantization (ARQ) for calibration data-free quantization error reduction, as well as (3) δ-Guided Bit Switching (δ-GBS) for adaptive bit-width allocation. Extensive experiments across multiple video generation benchmarks demonstrate that DVD-Quant achieves an approximately 2× speedup over full-precision baselines on advanced DiT models while maintaining visual fidelity. Notably, DVD-Quant is the first to enable W4A4 PTQ for Video DiTs without compromising video quality. Code and models will be available at https://github.com/lhxcs/DVD-Quant.

Citations

• Towards AI Search Paradigm
• SkyReels-V2: Infinite-length Film Generative Model
• Wan: Open and Advanced Large-Scale Video Generative Models
• Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200k
• QuantCache: Adaptive Importance-Guided Quantization with Hierarchical Latent and Layer Caching for Video Generation
• Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity
• Progressive Binarization with Semi-Structured Pruning for LLMs
• Open-Sora: Democratizing Efficient Video Production for All
• HunyuanVideo: A Systematic Framework For Large Video Generative Models
• Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model
• SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models
• Adaptive Caching for Faster Video Generation with Diffusion Transformers
• ARB-LLM: Alternating Refined Binarizations for Large Language Models
• SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration
• CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer
• Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers
• 2DQuant: Low-bit Post-Training Quantization for Image Super-Resolution
• ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation
• SpinQuant: LLM quantization with learned rotations
• PTQ4DiT: Post-training Quantization for Diffusion Transformers
• TerDiT: Ternary Diffusion Models with Transformers
• QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
• ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation
• QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks
• Latte: Latent Diffusion Transformer for Video Generation
• FRDiff : Feature Reuse for Universal Training-free Acceleration of Diffusion Models
• DeepCache: Accelerating Diffusion Models for Free
• VBench: Comprehensive Benchmark Suite for Video Generative Models
• Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
• OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models
• Q-Diffusion: Quantizing Diffusion Models
• Scalable Diffusion Models with Transformers
• Post-training Quantization on Diffusion Models
• SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
• Flow Matching for Generative Modeling
• Make-A-Video: Text-to-Video Generation without Text-Video Data
• All are Worth Words: A ViT Backbone for Diffusion Models
• Classifier-Free Diffusion Guidance
• Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
• Hierarchical Text-Conditional Image Generation with CLIP Latents
• Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
• Attention Is All You Need

Cited by

• Plug-and-Play Fidelity Optimization for Diffusion Transformer Acceleration via Cumulative Error Minimization
• InfVSR: Toward Consistency-Driven Streaming Generative Video Super-Resolution
• FlashEdit: Decoupling Speed, Structure, and Semantics for Precise Image Editing
• LRQ-DiT: Log-Rotation Post-Training Quantization of Diffusion Transformers for Image and Video Generation

Related

• Q-VDiT: Towards Accurate Quantization and Distillation of Video-Generation Diffusion Transformers
• ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation
• Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers
• VQ4DiT: Efficient Post-Training Vector Quantization for Diffusion Transformers
• LRQ-DiT: Log-Rotation Post-Training Quantization of Diffusion Transformers for Image and Video Generation