A Learnable Variational Model for Joint Multimodal MRI Reconstruction and Synthesis

2022/04/08 by Bian, Wanyu, Zhang, Qingchao, Ye, Xiaojing +1 · 1 citation
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #FOS: Mathematics #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #Optimization and Control (math.OC) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2204.03804

Abstract

Generating multi-contrasts/modal MRI of the same anatomy enriches diagnostic information but is limited in practice due to excessive data acquisition time. In this paper, we propose a novel deep-learning model for joint reconstruction and synthesis of multi-modal MRI using incomplete k-space data of several source modalities as inputs. The output of our model includes reconstructed images of the source modalities and high-quality image synthesized in the target modality. Our proposed model is formulated as a variational problem that leverages several learnable modality-specific feature extractors and a multimodal synthesis module. We propose a learnable optimization algorithm to solve this model, which induces a multi-phase network whose parameters can be trained using multi-modal MRI data. Moreover, a bilevel-optimization framework is employed for robust parameter training. We demonstrate the effectiveness of our approach using extensive numerical experiments.

Cited by

• Deep Unrolled Meta-Learning for Multi-Coil and Multi-Modality MRI with Adaptive Optimization

Related

• Multimodal MR Synthesis via Modality-Invariant Latent Representation
• Optimization-Based Deep learning methods for Magnetic Resonance Imaging Reconstruction and Synthesis
• Unified Cross-Modal Medical Image Synthesis with Hierarchical Mixture of Product-of-Experts
• Hi-Net: Hybrid-fusion Network for Multi-modal MR Image Synthesis
• Variational Multi-Task MRI Reconstruction: Joint Reconstruction,\n Registration and Super-Resolution