A generalizable 3D framework and model for self-supervised learning in medical imaging

2025/01/20 by Tony Xu, Xu, Tony, Sepehr Hosseini +11 · 10 citations
Computer Science · Engineering · Medicine · Psychology · #Artificial intelligence #Computer science #Machine learning #Medical Image Segmentation Techniques #Medical Imaging Techniques and Applications #Medical Imaging and Analysis #Medical imaging #Psychology

paper · pdf · doi:10.48550/arxiv.2501.11755

published in ArXiv.org

openalex publication_date 2025/01/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Current self-supervised learning methods for 3D medical imaging rely on simple pretext formulations and organ- or modality-specific datasets, limiting their generalizability and scalability. We present 3DINO, a cutting-edge SSL method adapted to 3D datasets, and use it to pretrain 3DINO-ViT: a general-purpose medical imaging model, on an exceptionally large, multimodal, and multi-organ dataset of ~100,000 3D medical imaging scans from over 10 organs. We validate 3DINO-ViT using extensive experiments on numerous medical imaging segmentation and classification tasks. Our results demonstrate that 3DINO-ViT generalizes across modalities and organs, including out-of-distribution tasks and datasets, outperforming state-of-the-art methods on the majority of evaluation metrics and labeled dataset sizes. Our 3DINO framework and 3DINO-ViT will be made available to enable research on 3D foundation models or further finetuning for a wide range of medical imaging applications.

Cited by

• Task-Aligned Self-Supervised Learning for Medical Image Analysis: A Task-Oriented Review with Practical Design Guidelines
• Revisiting 2D Foundation Models for Scalable 3D Medical Image Classification
• Rad-JEPA 3D: Radiology Joint-Embedding Predictive Model for 3D Computed Tomography
• Comprehensive language-image pre-training for 3D medical image understanding
• Shaken or Stirred? An Analysis of MetaFormer's Token Mixing for Medical Imaging
• Live(r) Die: Predicting Survival in Colorectal Liver Metastasis
• PRISM: A Framework Harnessing Unsupervised Visual Representations and Textual Prompts for Explainable MACE Survival Prediction from Cardiac Cine MRI
• Do Medical Foundation Models Generalize on the African Brain?
• PatTree: a novel approach for automated creation of multimodal, graph-based patient representations for medical classification tasks
• Evaluating Vision Language Models (VLMs) for Radiology: A Comprehensive Analysis
• Multi-encoder nnU-Net outperforms transformer models with self-supervised pretraining

Related

• 3D Self-Supervised Methods for Medical Imaging
• An OpenMind for 3D medical vision self-supervised learning
• UniMiSS: Universal Medical Self-Supervised Learning via Breaking Dimensionality Barrier
• Revisiting MAE pre-training for 3D medical image segmentation
• Cross-Dimensional Medical Self-Supervised Representation Learning Based on a Pseudo-3D Transformation