2022/01/26 by Wangbo Yu, Yu, Wangbo, Jinhao Du +7 · 1 citation
Computer Science · Mathematics · #Advanced Vision and Imaging #Artificial Intelligence (cs.AI) #Artificial intelligence #Artificial neural network #Autoencoder #Computer Graphics and Visualization Techniques #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Computer vision #Controllability #Deep learning #Encoder #FOS: Computer and information sciences #Focus (optics) #Generative Adversarial Networks and Image Synthesis #Image (mathematics) #Inpainting #Mathematics #Pattern recognition (psychology) #cs.AI #cs.CV
paper · pdf · doi:10.48550/arxiv.2201.10753
published in arXiv (Cornell University) (Cornell University)
arxiv created 2022/01/26 · openalex publication_date 2022/01/26 · arxiv updated 2022/01/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/04
Image inpainting approaches have achieved significant progress with the help of deep neural networks. However, existing approaches mainly focus on leveraging the priori distribution learned by neural networks to produce a single inpainting result or further yielding multiple solutions, where the controllability is not well studied. This paper develops a novel image inpainting approach that enables users to customize the inpainting result by their own preference or memory. Specifically, our approach is composed of two stages that utilize the prior of neural network and user's guidance to jointly inpaint corrupted images. In the first stage, an autoencoder based on a novel external spatial attention mechanism is deployed to produce reconstructed features of the corrupted image and a coarse inpainting result that provides semantic mask as the medium for user interaction. In the second stage, a semantic decoder that takes the reconstructed features as prior is adopted to synthesize a fine inpainting result guided by user's customized semantic mask, so that the final inpainting result will share the same content with user's guidance while the textures and colors reconstructed in the first stage are preserved. Extensive experiments demonstrate the superiority of our approach in terms of inpainting quality and controllability.