Learning to Match 2D Keypoints Across Preoperative MR and Intraoperative Ultrasound

2024/09/12 by Hassan Rasheed, Rasheed, Hassan, Reuben Dorent +15 · 2 citations
Medicine · #Colorectal Cancer Screening and Detection #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Radiomics and Machine Learning in Medical Imaging #Surgical Simulation and Training

paper · pdf · doi:10.48550/arxiv.2409.08169

openalex publication_date 2024/09/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose in this paper a texture-invariant 2D keypoints descriptor specifically designed for matching preoperative Magnetic Resonance (MR) images with intraoperative Ultrasound (US) images. We introduce a matching-by-synthesis strategy, where intraoperative US images are synthesized from MR images accounting for multiple MR modalities and intraoperative US variability. We build our training set by enforcing keypoints localization over all images then train a patient-specific descriptor network that learns texture-invariant discriminant features in a supervised contrastive manner, leading to robust keypoints descriptors. Our experiments on real cases with ground truth show the effectiveness of the proposed approach, outperforming the state-of-the-art methods and achieving 80.35% matching precision on average.

Cited by

• Deep Biomechanically-Guided Interpolation for Keypoint-Based Brain Shift Registration
• A 3D Cross-modal Keypoint Descriptor for MR-US Matching and Registration

Related

• A 3D Cross-modal Keypoint Descriptor for MR-US Matching and Registration
• Global Multi-modal 2D/3D Registration via Local Descriptors Learning
• Image Retrieval with Intra-Sweep Representation Learning for Neck Ultrasound Scanning Guidance
• Coarse-to-Fine Joint Registration of MR and Ultrasound Images via Imaging Style Transfer
• End-to-end Ultrasound Frame to Volume Registration