Learning General-Purpose Biomedical Volume Representations using Randomized Synthesis

2024/11/04 by Neel Dey, Dey, Neel, Benjamin Billot +13 · 9 citations
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Digital Imaging for Blood Diseases #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2411.02372

openalex publication_date 2024/11/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Current volumetric biomedical foundation models struggle to generalize as public 3D datasets are small and do not cover the broad diversity of medical procedures, conditions, anatomical regions, and imaging protocols. We address this by creating a representation learning method that instead anticipates strong domain shifts at training time itself. We first propose a data engine that synthesizes highly variable training samples that would enable generalization to new biomedical contexts. To then train a single 3D network for any voxel-level task, we develop a contrastive learning method that pretrains the network to be stable against nuisance imaging variation simulated by the data engine, a key inductive bias for generalization. This network's features can be used as robust representations of input images for downstream tasks and its weights provide a strong, dataset-agnostic initialization for finetuning on new datasets. As a result, we set new standards across both multimodality registration and few-shot segmentation, a first for any 3D biomedical vision model, all without (pre-)training on any existing dataset of real images.

Cited by

• Patient-Agnostic Synthetic Pretraining for Efficient Patient-Specific Intraoperative 2D/3D Registration
• Pancakes: Consistent Multi-Protocol Image Segmentation Across Biomedical Domains
• Deep infant brain segmentation from multi-contrast MRI
• Anatomica: Localized Control over Geometric and Topological Properties for Anatomical Diffusion Models
• A Scalable Distributed Framework for Multimodal GigaVoxel Image Registration
• Robust Fetal Pose Estimation across Gestational Ages via Cross-Population Augmentation
• Surrogate Supervision for Robust and Generalizable Deformable Image Registration
• Does DINOv3 Set a New Medical Vision Standard? Benchmarking 2D and 3D Classification, Segmentation, and Registration
• CardioComposer: Leveraging Differentiable Geometry for Compositional Control of Anatomical Diffusion Models

Related

• MedContext: Learning Contextual Cues for Efficient Volumetric Medical Segmentation
• A generalizable 3D framework and model for self-supervised learning in medical imaging
• Momentum Contrastive Voxel-wise Representation Learning for Semi-supervised Volumetric Medical Image Segmentation
• Domain-randomized deep learning for neuroimage analysis
• Towards Robust In-Context Learning for Medical Image Segmentation via Data Synthesis