2025/05/20 by Xiangyu Yue, Liu, Wenze, Yue, Xiangyu · 5 citations
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Metaheuristic Optimization Algorithms Research
paper · pdf · doi:10.48550/arxiv.2505.14502
openalex publication_date 2025/05/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
To accelerate diffusion model inference, numerical solvers perform poorly at extremely small steps, while distillation techniques often introduce complexity and instability. This work presents an intermediate strategy, balancing performance and cost, by learning ODE integration using loss functions derived from the derivative-integral relationship, inspired by Monte Carlo integration and Picard iteration. From a geometric perspective, the losses operate by gradually extending the tangent to the secant, thus are named as secant losses. The target of secant losses is the same as that of diffusion models, or the diffusion model itself, leading to great training stability. By fine-tuning or distillation, the secant version of EDM achieves a 10-step FID of 2.14 on CIFAR-10, while the secant version of SiT-XL/2 attains a 4-step FID of 2.27 and an 8-step FID of 1.96 on ImageNet-256×256. Code is available at https://github.com/poppuppy/secant-expectation.