2025/02/11 by Zheng Fang, Fang, Zheng, Lichuan Xiang +8 · 1 voice
Computer Science · #Advanced Image and Video Retrieval Techniques #FOS: Computer and information sciences #Graph Theory and Algorithms #Graphics (cs.GR) #Machine Learning (cs.LG) #Medical Image Segmentation Techniques #cs.GR #cs.LG
paper · pdf · doi:10.48550/arxiv.2502.10451
openalex publication_date 2025/02/11 · arxiv published 2025/02/11 · arxiv updated 2025/02/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
ControlNet offers a powerful way to guide diffusion-based generative models, yet most implementations rely on ad-hoc heuristics to choose which network blocks to control-an approach that varies unpredictably with different tasks. To address this gap, we propose FlexControl, a novel framework that copies all diffusion blocks during training and employs a trainable gating mechanism to dynamically select which blocks to activate at each denoising step. With introducing a computation-aware loss, we can encourage control blocks only to activate when it benefit the generation quality. By eliminating manual block selection, FlexControl enhances adaptability across diverse tasks and streamlines the design pipeline, with computation-aware training loss in an end-to-end training manner. Through comprehensive experiments on both UNet (e.g., SD1.5) and DiT (e.g., SD3.0), we show that our method outperforms existing ControlNet variants in certain key aspects of interest. As evidenced by both quantitative and qualitative evaluations, FlexControl preserves or enhances image fidelity while also reducing computational overhead by selectively activating the most relevant blocks. These results underscore the potential of a flexible, data-driven approach for controlled diffusion and open new avenues for efficient generative model design. The code will soon be available at https://github.com/Anonymousuuser/FlexControl.