2022/12/13 by Qinghe Wang, Lijie Liu, Wang, Qinghe +11 · 2 citations
Computer Science · Engineering · Medicine · #AI in cancer detection #Anatomy and Medical Technology #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Fetal and Pediatric Neurological Disorders
paper · pdf · doi:10.48550/arxiv.2212.06458
openalex publication_date 2022/12/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Image-based head swapping task aims to stitch a source head to another source body flawlessly. This seldom-studied task faces two major challenges: 1) Preserving the head and body from various sources while generating a seamless transition region. 2) No paired head swapping dataset and benchmark so far. In this paper, we propose a semantic-mixing diffusion model for head swapping (HS-Diffusion) which consists of a latent diffusion model (LDM) and a semantic layout generator. We blend the semantic layouts of source head and source body, and then inpaint the transition region by the semantic layout generator, achieving a coarse-grained head swapping. Semantic-mixing LDM can further implement a fine-grained head swapping with the inpainted layout as condition by a progressive fusion process, while preserving head and body with high-quality reconstruction. To this end, we propose a semantic calibration strategy for natural inpainting and a neck alignment for geometric realism. Importantly, we construct a new image-based head swapping benchmark and design two tailor-designed metrics (Mask-FID and Focal-FID). Extensive experiments demonstrate the superiority of our framework. The code will be available: https://github.com/qinghew/HS-Diffusion.