2023/12/14 by Enis Simsar, Alessio Tonioni, Simsar, Enis +7 · 1 voice · 7 citations
Computer Science · Mathematics · Medicine · #Advanced Neuroimaging Techniques and Applications #Artificial intelligence #Cluster analysis #Computer science #Computer vision #Controllability #Focus (optics) #Generative Adversarial Networks and Image Synthesis #Image (mathematics) #Image editing #Mathematics #Noise reduction #Pattern recognition (psychology) #Regularization (linguistics) #Segmentation #cs.CV
paper · pdf · doi:10.48550/arxiv.2312.09256
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2023/12/14 · openalex created_date 2023/12/16 · openalex updated_date 2026/07/28
Diffusion models (DMs) have gained prominence due to their ability to generate high-quality varied images with recent advancements in text-to-image generation. The research focus is now shifting towards the controllability of DMs. A significant challenge within this domain is localized editing, where specific areas of an image are modified without affecting the rest of the content. This paper introduces LIME for localized image editing in diffusion models. LIME does not require user-specified regions of interest (RoI) or additional text input, but rather employs features from pre-trained methods and a straightforward clustering method to obtain precise editing mask. Then, by leveraging cross-attention maps, it refines these segments for finding regions to obtain localized edits. Finally, we propose a novel cross-attention regularization technique that penalizes unrelated cross-attention scores in the RoI during the denoising steps, ensuring localized edits. Our approach, without re-training, fine-tuning and additional user inputs, consistently improves the performance of existing methods in various editing benchmarks. The project page can be found at https://enisimsar.github.io/LIME/.