TomoGraphView: 3D Medical Image Classification with Omnidirectional Slice Representations and Graph Neural Networks
2025/11/12 by Kiechle, Johannes, Fischer, Stefan M., Lang, Daniel M. +5
#Artificial Intelligence (cs.AI) #FOS: Biological sciences #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #Quantitative Methods (q-bio.QM) #electronic engineering #information engineering
paper · doi:10.48550/arxiv.2511.09605
Abstract
The sharp rise in medical tomography examinations has created a demand for automated systems that can reliably extract informative features for downstream tasks such as tumor characterization. Although 3D volumes contain richer information than individual slices, effective 3D classification remains difficult: volumetric data encode complex spatial dependencies, and the scarcity of large-scale 3D datasets has constrained progress toward 3D foundation models. As a result, many recent approaches rely on 2D vision foundation models trained on natural images, repurposing them as feature extractors for medical scans with surprisingly strong performance. Despite their practical success, current methods that apply 2D foundation models to 3D scans via slice-based decomposition remain fundamentally limited. Standard slicing along axial, sagittal, and coronal planes often fails to capture the true spatial extent of a structure when its orientation does not align with these canonical views. More critically, most approaches aggregate slice features independently, ignoring the underlying 3D geometry and losing spatial coherence across slices. To overcome these limitations, we propose TomoGraphView, a novel framework that integrates omnidirectional volume slicing with spherical graph-based feature aggregation. Instead of restricting the model to axial, sagittal, or coronal planes, our method samples both canonical and non-canonical cross-sections generated from uniformly distributed points on a sphere enclosing the volume. We publicly share our accessible code base at http://github.com/compai-lab/2025-MedIA-kiechle and provide a user-friendly library for omnidirectional volume slicing at https://pypi.org/project/OmniSlicer.
Citations
- Structured Spectral Graph Representation Learning for Multi-label Abnormality Analysis from 3D CT Scans
- Structured Spectral Graph Learning for Anomaly Classification in 3D Chest CT Scans
- Franca: Nested Matryoshka Clustering for Scalable Visual Representation Learning
- Multimodal Large Language Models for Medicine: A Comprehensive Survey
- SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features
- An OpenMind for 3D medical vision self-supervised learning
- Phikon-v2, A large and public feature extractor for biomarker prediction
- Graph Neural Networks: A suitable Alternative to MLPs in Latent 3D Medical Image Classification?
- VISTA3D: A Unified Segmentation Foundation Model For 3D Medical Imaging
- DinoBloom: A Foundation Model for Generalizable Cell Embeddings in Hematology
- DINOv2 based Self Supervised Learning For Few Shot Medical Image Segmentation
- Comparative Analysis of ImageNet Pre-Trained Deep Learning Models and DINOv2 in Medical Imaging Classification
- Evaluating General Purpose Vision Foundation Models for Medical Image Analysis: An Experimental Study of DINOv2 on Radiology Benchmarks
- Are Natural Domain Foundation Models Useful for Medical Image Classification?
- Foundational Models in Medical Imaging: A Comprehensive Survey and Future Vision
- DINOv2: Learning Robust Visual Features without Supervision
- Segment Everything Everywhere All at Once
- Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation
- Emerging Properties in Self-Supervised Vision Transformers
- Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
- Learning Transferable Visual Models From Natural Language Supervision
- Principal Neighbourhood Aggregation for Graph Nets
- A Simple Framework for Contrastive Learning of Visual Representations
- A CNN-LSTM Architecture for Detection of Intracranial Hemorrhage on CT scans
- Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysis
- The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes
- How Powerful are Graph Neural Networks?
- Computational Radiomics System to Decode the Radiographic Phenotype
- Inductive Representation Learning on Large Graphs
- Neural Message Passing for Quantum Chemistry
- U-Net: Convolutional Networks for Biomedical Image Segmentation
- Long-term Recurrent Convolutional Networks for Visual Recognition and Description
- Does DINOv3 Set a New Medical Vision Standard? Benchmarking 2D and 3D Classification, Segmentation, and Registration
- A Comprehensive Survey on Graph Neural Networks
Related