Adaptive Sampling Scheduler

2025/09/16 by Wang Qi, Wang, Qi, Zhu, Shuliang +2
Computer Science · Decision Sciences · Engineering · #Adaptive sampling #Advanced Bandit Algorithms Research #Consistency (knowledge bases) #Data Stream Mining Techniques #Distillation #Flexibility (engineering) #Process (computing) #Reservoir Engineering and Simulation Methods #Sampling (signal processing)

paper · pdf · doi:10.48550/arxiv.2509.12569

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2025/09/16 · openalex created_date 2025/10/18 · openalex updated_date 2026/08/05

Abstract

Consistent distillation methods have evolved into effective techniques that significantly accelerate the sampling process of diffusion models. Although existing methods have achieved remarkable results, the selection of target timesteps during distillation mainly relies on deterministic or stochastic strategies, which often require sampling schedulers to be designed specifically for different distillation processes. Moreover, this pattern severely limits flexibility, thereby restricting the full sampling potential of diffusion models in practical applications. To overcome these limitations, this paper proposes an adaptive sampling scheduler that is applicable to various consistency distillation frameworks. The scheduler introduces three innovative strategies: (i) dynamic target timestep selection, which adapts to different consistency distillation frameworks by selecting timesteps based on their computed importance; (ii) Optimized alternating sampling along the solution trajectory by guiding forward denoising and backward noise addition based on the proposed time step importance, enabling more effective exploration of the solution space to enhance generation performance; and (iii) Utilization of smoothing clipping and color balancing techniques to achieve stable and high-quality generation results at high guidance scales, thereby expanding the applicability of consistency distillation models in complex generation scenarios. We validated the effectiveness and flexibility of the adaptive sampling scheduler across various consistency distillation methods through comprehensive experimental evaluations. Experimental results consistently demonstrated significant improvements in generative performance, highlighting the strong adaptability achieved by our method.

Citations

• Target-Driven Distillation: Consistency Distillation with Target Timestep Selection and Decoupled Guidance
• Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion
• SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
• On Distillation of Guided Diffusion Models
• DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps
• Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
• Score-Based Generative Modeling through Stochastic Differential Equations
• Denoising Diffusion Implicit Models
• Denoising Diffusion Probabilistic Models
• Generative Modeling by Estimating Gradients of the Data Distribution
• A Style-Based Generator Architecture for Generative Adversarial Networks
• Improved Techniques for Training GANs
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics
• Microsoft COCO: Common Objects in Context

Related

• Target-Driven Distillation: Consistency Distillation with Target Timestep Selection and Decoupled Guidance
• Image-Free Timestep Distillation via Continuous-Time Consistency with Trajectory-Sampled Pairs
• Inference-Time Diffusion Model Distillation
• Sampler Scheduler for Diffusion Models
• Learnable Sampler Distillation for Discrete Diffusion Models