Unsupervised learning for cross-domain medical image synthesis using deformation invariant cycle consistency networks

2018/08/12 by Wang, Chengjia, Macnaught, Gillian, Papanastasiou, Giorgos +2
#Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.1808.03944

Abstract

Recently, the cycle-consistent generative adversarial networks (CycleGAN) has been widely used for synthesis of multi-domain medical images. The domain-specific nonlinear deformations captured by CycleGAN make the synthesized images difficult to be used for some applications, for example, generating pseudo-CT for PET-MR attenuation correction. This paper presents a deformation-invariant CycleGAN (DicycleGAN) method using deformable convolutional layers and new cycle-consistency losses. Its robustness dealing with data that suffer from domain-specific nonlinear deformations has been evaluated through comparison experiments performed on a multi-sequence brain MR dataset and a multi-modality abdominal dataset. Our method has displayed its ability to generate synthesized data that is aligned with the source while maintaining a proper quality of signal compared to CycleGAN-generated data. The proposed model also obtained comparable performance with CycleGAN when data from the source and target domains are alignable through simple affine transformations.

Related

• Cross Modality Medical Image Synthesis for Improving Liver Segmentation
• Deformation-aware GAN for Medical Image Synthesis with Substantially Misaligned Pairs
• Generative Adversarial Networks for MR-CT Deformable Image Registration
• Unpaired Brain MR-to-CT Synthesis using a Structure-Constrained CycleGAN
• Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Network