Efficient Score Pre-computation for Diffusion Models via Cross-Matrix Krylov Projection

2025/11/19 by Lau, Kaikwan, Na, Andrew S., Wan, Justin W. L.
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2511.17634

Abstract

This paper presents a novel framework to accelerate score-based diffusion models. It first converts the standard stable diffusion model into the Fokker-Planck formulation which results in solving large linear systems for each image. For training involving many images, it can lead to a high computational cost. The core innovation is a cross-matrix Krylov projection method that exploits mathematical similarities between matrices, using a shared subspace built from ``seed" matrices to rapidly solve for subsequent ``target" matrices. Our experiments show that this technique achieves a 15.8% to 43.7% time reduction over standard sparse solvers. Additionally, we compare our method against DDPM baselines in denoising tasks, showing a speedup of up to 115×. Furthermore, under a fixed computational budget, our model is able to produce high-quality images while DDPM fails to generate recognizable content, illustrating our approach is a practical method for efficient generation in resource-limited settings.

Citations

• Efficient Denoising using Score Embedding in Score-based Diffusion Models
• Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference
• Probability flow solution of the Fokker-Planck equation
• DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps
• Score-Based Generative Modeling through Stochastic Differential Equations
• Denoising Diffusion Implicit Models
• Efficient Learning of Generative Models via Finite-Difference Score Matching
• Denoising Diffusion Probabilistic Models
• Improved Techniques for Training Score-Based Generative Models
• Generative Modeling by Estimating Gradients of the Data Distribution
• Attention Is All You Need
• U-Net: Convolutional Networks for Biomedical Image Segmentation
• Deep Learning Face Attributes in the Wild

Related

• Efficient Denoising using Score Embedding in Score-based Diffusion Models
• Gotta Go Fast When Generating Data with Score-Based Models
• A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models
• Optimization-Free Diffusion Model -- A Perturbation Theory Approach
• Accelerating Score-based Generative Models with Preconditioned Diffusion Sampling