2023/04/22 by Guoqiang Zhang, Kenta Niwa, Zhang, Guoqiang +3 · 2 citations
Computer Science · Mathematics · Physics and Astronomy · #Algorithm #Applied mathematics #Computer science #Differential equation #FOS: Computer and information sciences #FOS: Mathematics #Fractional Differential Equations Solutions #Function (biology) #Machine Learning (cs.LG) #Mathematical analysis #Mathematical optimization #Mathematics #Model Reduction and Neural Networks #Neural Networks and Applications #Numerical Analysis (math.NA) #Ode #Ordinary differential equation #Sampling (signal processing) #Solver
paper · pdf · doi:10.48550/arxiv.2304.11328
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2023/04/22 · openalex created_date 2023/04/27 · openalex updated_date 2026/07/28
A popular approach to sample a diffusion-based generative model is to solve an ordinary differential equation (ODE). In existing samplers, the coefficients of the ODE solvers are pre-determined by the ODE formulation, the reverse discrete timesteps, and the employed ODE methods. In this paper, we consider accelerating several popular ODE-based sampling processes (including EDM, DDIM, and DPM-Solver) by optimizing certain coefficients via improved integration approximation (IIA). We propose to minimize, for each time step, a mean squared error (MSE) function with respect to the selected coefficients. The MSE is constructed by applying the original ODE solver for a set of fine-grained timesteps, which in principle provides a more accurate integration approximation in predicting the next diffusion state. The proposed IIA technique does not require any change of a pre-trained model, and only introduces a very small computational overhead for solving a number of quadratic optimization problems. Extensive experiments show that considerably better FID scores can be achieved by using IIA-EDM, IIA-DDIM, and IIA-DPM-Solver than the original counterparts when the neural function evaluation (NFE) is small (i.e., less than 25).