2024/07/17 by Jiankun Zhao, Bowen Song, Zhao, Jiankun +3 · 6 citations
Computer Science · Engineering · Mathematics · Medicine · #Applied mathematics #Artificial intelligence #BIM and Construction Integration #Calculus (dental) #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Consistency (knowledge bases) #Econometrics #FOS: Computer and information sciences #FOS: Electrical engineering #Geometry #Image and Video Processing (eess.IV) #Inverse #Manufacturing Process and Optimization #Mathematical economics #Mathematical optimization #Mathematics #Medicine #Neural Networks and Applications #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2407.12676
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2024/07/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Diffusion models have been demonstrated as strong priors for solving general inverse problems. Most existing Diffusion model-based Inverse Problem Solvers (DIS) employ a plug-and-play approach to guide the sampling trajectory with either projections or gradients. Though effective, these methods generally necessitate hundreds of sampling steps, posing a dilemma between inference time and reconstruction quality. In this work, we try to push the boundary of inference steps to 1-2 NFEs while still maintaining high reconstruction quality. To achieve this, we propose to leverage a pretrained distillation of diffusion model, namely consistency model, as the data prior. The key to achieving few-step guidance is to enforce two types of constraints during the sampling process of the consistency model: soft measurement constraint with ControlNet and hard measurement constraint via optimization. Supporting both single-step reconstruction and multistep refinement, the proposed framework further provides a way to trade image quality with additional computational cost. Within comparable NFEs, our method achieves new state-of-the-art in diffusion-based inverse problem solving, showcasing the significant potential of employing prior-based inverse problem solvers for real-world applications. Code is available at: https://github.com/BioMed-AI-Lab-U-Michgan/cosign.