2025/02/09 by Idan Achituve, Hai Victor Habi, Achituve, Idan +9 · 2 citations
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Face and Expression Recognition #Gaussian Processes and Bayesian Inference #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #Neural Networks and Applications #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2502.05908
openalex publication_date 2025/02/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In image processing, solving inverse problems is the task of finding plausible reconstructions of an image that was corrupted by some (usually known) degradation operator. Commonly, this process is done using a generative image model that can guide the reconstruction towards solutions that appear natural. The success of diffusion models over the last few years has made them a leading candidate for this task. However, the sequential nature of diffusion models makes this conditional sampling process challenging. Furthermore, since diffusion models are often defined in the latent space of an autoencoder, the encoder-decoder transformations introduce additional difficulties. To address these challenges, we suggest a novel sampling method based on sequential Monte Carlo (SMC) in the latent space of diffusion models. We name our method LD-SMC. We define a generative model for the data using additional auxiliary observations and perform posterior inference with SMC sampling based on a reverse diffusion process. Empirical evaluations on ImageNet and FFHQ show the benefits of LD-SMC over competing methods in various inverse problem tasks and especially in challenging inpainting tasks.