2025/02/05 by Yunlong Yuan, Yuanfan Guo, Yuan, Yunlong +9 · 1 citation
Computer Science · #FOS: Computer and information sciences #FOS: Electrical engineering #Graphics (cs.GR) #Image and Signal Denoising Methods #Image and Video Processing (eess.IV) #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2502.03496
openalex publication_date 2025/02/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Text-driven video generation has advanced significantly due to developments in diffusion models. Beyond the training and sampling phases, recent studies have investigated noise priors of diffusion models, as improved noise priors yield better generation results. One recent approach employs the Fourier transform to manipulate noise, marking the initial exploration of frequency operations in this context. However, it often generates videos that lack motion dynamics and imaging details. In this work, we provide a comprehensive theoretical analysis of the variance decay issue present in existing methods, contributing to the loss of details and motion dynamics. Recognizing the critical impact of noise distribution on generation quality, we introduce FreqPrior, a novel noise initialization strategy that refines noise in the frequency domain. Our method features a novel filtering technique designed to address different frequency signals while maintaining the noise prior distribution that closely approximates a standard Gaussian distribution. Additionally, we propose a partial sampling process by perturbing the latent at an intermediate timestep during finding the noise prior, significantly reducing inference time without compromising quality. Extensive experiments on VBench demonstrate that our method achieves the highest scores in both quality and semantic assessments, resulting in the best overall total score. These results highlight the superiority of our proposed noise prior.