Block Cascading: Training Free Acceleration of Block-Causal Video Models

2025/11/25 by Bandyopadhyay, Hmrishav, Pinnaparaju, Nikhil, Entezari, Rahim +3
Computer Science · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis

paper · doi:10.48550/arxiv.2511.20426

openalex publication_date 2025/11/25 · openalex created_date 2025/11/28 · openalex updated_date 2026/07/28

Abstract

Block-causal video generation faces a stark speed-quality trade-off: small 1.3B models manage only 16 FPS while large 14B models crawl at 4.5 FPS, forcing users to choose between responsiveness and quality. Block Cascading significantly mitigates this trade-off through training-free parallelization. Our key insight: future video blocks do not need fully denoised current blocks to begin generation. By starting block generation with partially denoised context from predecessors, we transform sequential pipelines into parallel cascades where multiple blocks denoise simultaneously. With 5 GPUs exploiting temporal parallelism, we achieve ~2x acceleration across all model scales: 1.3B models accelerate from 16 to 30 FPS, 14B models from 4.5 to 12.5 FPS. Beyond inference speed, Block Cascading eliminates overhead from KV-recaching (of ~200ms) during context switches for interactive generation. Extensive evaluations validated against multiple block-causal pipelines demonstrate no significant loss in generation quality when switching from block-causal to Block Cascading pipelines for inference. Project Page: https://hmrishavbandy.github.io/block_cascading_page/

Citations

• MotionStream: Real-Time Video Generation with Interactive Motion Controls
• Rolling Forcing: Autoregressive Long Video Diffusion in Real Time
• SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer
• LongLive: Real-time Interactive Long Video Generation
• KVSink: Understanding and Enhancing the Preservation of Attention Sinks in KV Cache Quantization for LLMs
• Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
• Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generation
• Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion
• VSA: Faster Video Diffusion with Trainable Sparse Attention
• MAGI-1: Autoregressive Video Generation at Scale
• SkyReels-V2: Infinite-length Film Generative Model
• Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Models
• Wan: Open and Advanced Large-Scale Video Generative Models
• SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillation
• Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model
• Fast Video Generation with Sliding Tile Attention
• GameFactory: Creating New Games with Generative Interactive Videos
• Diffusion Adversarial Post-Training for One-Step Video Generation
• Qwen2.5 Technical Report
• From Slow Bidirectional to Fast Autoregressive Video Diffusion Models
• HunyuanVideo: A Systematic Framework For Large Video Generative Models
• NitroFusion: High-Fidelity Single-Step Diffusion through Dynamic Adversarial Training
• Movie Gen: A Cast of Media Foundation Models
• OSV: One Step is Enough for High-Quality Image to Video Generation
• CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer
• FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision
• Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion
• ViD-GPT: Introducing GPT-style Autoregressive Generation in Video Diffusion Models
• SF-V: Single Forward Video Generation Model
• Improved Distribution Matching Distillation for Fast Image Synthesis
• Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation
• Scaling Rectified Flow Transformers for High-Resolution Image Synthesis
• SDXL-Lightning: Progressive Adversarial Diffusion Distillation
• KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
• One-step Diffusion with Distribution Matching Distillation
• ART\boldsymbol⋅V: Auto-Regressive Text-to-Video Generation with Diffusion Models
• VBench: Comprehensive Benchmark Suite for Video Generative Models
• Ring Attention with Blockwise Transformers for Near-Infinite Context
• Efficient Streaming Language Models with Attention Sinks
• StyleInV: A Temporal Style Modulated Inversion Network for Unconditional Video Generation
• ModelScope Text-to-Video Technical Report
• ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation
• Efficiently Scaling Transformer Inference
• DreamFusion: Text-to-3D using 2D Diffusion
• StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2
• High-Resolution Image Synthesis with Latent Diffusion Models
• A Good Image Generator Is What You Need for High-Resolution Video Synthesis
• VideoGPT: Video Generation using VQ-VAE and Transformers
• Score-Based Generative Modeling through Stochastic Differential Equations
• Denoising Diffusion Probabilistic Models
• Conv-Linformer: Boosting Linformer's Performance with Convolution in Small-Scale Settings
• Language Models are Few-Shot Learners
• Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
• Attention Is All You Need

Related

• BIFE: Better Interaction, Fewer Errors for Minute-Long Video Generation
• Ms. Forcing: Efficient Streaming Video Generation with Multi-Scale Patchification and Attention
• ACID: Adaptive Caching for vIDeo generation
• Seedance 1.0: Exploring the Boundaries of Video Generation Models
• LightCache: Memory-Efficient, Training-Free Acceleration for Video Generation