Consistent View Alignment Improves Foundation Models for 3D Medical Image Segmentation

2025/09/17 by Puru Vaish, Felix J. Meister, Vaish, Puru +7
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Convolutional neural network #Domain Adaptation and Few-Shot Learning #External Data Representation #FOS: Computer and information sciences #Feature learning #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Point (geometry) #Representation (politics) #Segmentation #Synthetic data #Transformer

paper · pdf · doi:10.48550/arxiv.2509.13846

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2025/09/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Many recent approaches in representation learning implicitly assume that uncorrelated views of a data point are sufficient to learn meaningful representations for various downstream tasks. In this work, we challenge this assumption and demonstrate that meaningful structure in the latent space does not emerge naturally. Instead, it must be explicitly induced. We propose a method that aligns representations from different views of the data to align complementary information without inducing false positives. Our experiments show that our proposed self-supervised learning method, Consistent View Alignment, improves performance for downstream tasks, highlighting the critical role of structured view alignment in learning effective representations. Our method achieved first and second place in the MICCAI 2025 SSL3D challenge when using a Primus vision transformer and ResEnc convolutional neural network, respectively. The code and pretrained model weights are released at https://github.com/Tenbatsu24/LatentCampus.

Citations

• Primus: Enforcing Attention Usage for 3D Medical Image Segmentation
• An OpenMind for 3D medical vision self-supervised learning
• nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentation
• VoCo: A Simple-yet-Effective Volume Contrastive Learning Framework for 3D Medical Image Analysis
• Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled video
• MIS-FM: 3D Medical Image Segmentation using Foundation Models Pretrained on a Large-Scale Unannotated Dataset
• DINOv2: Learning Robust Visual Features without Supervision
• Robust Contrastive Learning against Noisy Views
• Masked Autoencoders Are Scalable Vision Learners
• Emerging Properties in Self-Supervised Vision Transformers
• Escaping the Big Data Paradigm with Compact Transformers
• Exploring Simple Siamese Representation Learning
• Exploring Simple Siamese Representation Learning
• Intriguing Properties of Contrastive Losses
• wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
• Bootstrap your own latent: A new approach to self-supervised Learning
• A Simple Framework for Contrastive Learning of Visual Representations
• Symmetric Cross Entropy for Robust Learning with Noisy Labels
• Invariant Risk Minimization
• Wasserstein Dependency Measure for Representation Learning
• Representation Learning with Contrastive Predictive Coding
• Learning from Noisy Labels with Distillation

Related

• Self-Supervised Alignment Learning for Medical Image Segmentation
• BARL: Bilateral Alignment in Representation and Label Spaces for Semi-Supervised Volumetric Medical Image Segmentation
• Anatomical Invariance Modeling and Semantic Alignment for Self-supervised Learning in 3D Medical Image Analysis
• How do Cross-View and Cross-Modal Alignment Affect Representations in Contrastive Learning?
• T3D: Advancing 3D Medical Vision-Language Pre-training by Learning Multi-View Visual Consistency