Whole Slide Images are 2D Point Clouds: Context-Aware Survival Prediction using Patch-based Graph Convolutional Networks

2021/07/27 by Richard J. Chen, Ming Y. Lu, Chen, Richard J. +12 · 27 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · Engineering · Medicine · #AI in cancer detection #Cancer Genomics and Diagnostics #Computer Vision and Pattern Recognition (cs.CV) #FOS: Biological sciences #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Video Processing (eess.IV) #Radiomics and Machine Learning in Medical Imaging #Tissues and Organs (q-bio.TO) #cs.CV #eess.IV #electronic engineering #information engineering #q-bio.TO

paper · pdf · doi:10.48550/arxiv.2107.13048

MICCAI 2021

arxiv created 2021/07/27 · openalex publication_date 2021/07/27 · arxiv updated 2021/07/29 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28

Abstract

Cancer prognostication is a challenging task in computational pathology that requires context-aware representations of histology features to adequately infer patient survival. Despite the advancements made in weakly-supervised deep learning, many approaches are not context-aware and are unable to model important morphological feature interactions between cell identities and tissue types that are prognostic for patient survival. In this work, we present Patch-GCN, a context-aware, spatially-resolved patch-based graph convolutional network that hierarchically aggregates instance-level histology features to model local- and global-level topological structures in the tumor microenvironment. We validate Patch-GCN with 4,370 gigapixel WSIs across five different cancer types from the Cancer Genome Atlas (TCGA), and demonstrate that Patch-GCN outperforms all prior weakly-supervised approaches by 3.58-9.46%. Our code and corresponding models are publicly available at https://github.com/mahmoodlab/Patch-GCN.

Cited by

• StainNet: Scaling Self-Supervised Foundation Models on Immunohistochemistry and Special Stains for Computational Pathology
• Structural Prognostic Event Modeling for Multimodal Cancer Survival Analysis
• Together, Then Apart: Balancing Alignment and Distinctiveness for Multimodal Survival Analysis
• Spatially-Aware Mixture of Experts with Log-Logistic Survival Modeling for Whole-Slide Images
• ITC-RWKV: Interactive Tissue-Cell Modeling with Recurrent Key-Value Aggregation for Histopathological Subtyping
• DCMIL: A Progressive Representation Learning of Whole Slide Images for Cancer Prognosis Analysis
• A Hierarchical Geometry-guided Transformer for Histological Subtyping of Primary Liver Cancer
• SlideMamba: Entropy-Based Adaptive Fusion of GNN and Mamba for Enhanced Representation Learning in Digital Pathology
• C²MIL: Synchronizing Semantic and Topological Causalities in Multiple Instance Learning for Robust and Interpretable Survival Analysis
• PS3: A Multimodal Transformer Integrating Pathology Reports with Histology Images and Biological Pathways for Cancer Survival Prediction
• Enhancing WSI-Based Survival Analysis with Report-Auxiliary Self-Distillation
• torchmil: A PyTorch-based library for deep Multiple Instance Learning
• Cross-Cancer Knowledge Transfer in WSI-based Prognosis Prediction
• IPGPhormer: Interpretable Pathology Graph-Transformer for Survival Analysis
• PTCMIL: Multiple Instance Learning via Prompt Token Clustering for Whole Slide Image Analysis
• Survival Modeling from Whole Slide Images via Patch-Level Graph Clustering and Mixture Density Experts
• Cracking Instance Jigsaw Puzzles: An Alternative to Multiple Instance Learning for Whole Slide Image Analysis
• HiLa: Hierarchical Vision-Language Collaboration for Cancer Survival Prediction
• MiCo: Multiple Instance Learning with Context-Aware Clustering for Whole Slide Image Analysis
• PathGene: Benchmarking Driver Gene Mutations and Exon Prediction Using Multicenter Lung Cancer Histopathology Image Dataset
• Hypergraph Mamba for Efficient Whole Slide Image Understanding
• Exploiting Low-Dimensional Manifold of Features for Few-Shot Whole Slide Image Classification
• Denoising Mutual Knowledge Distillation in Bi-Directional Multiple Instance Learning
• Multimodal Cancer Survival Analysis via Hypergraph Learning with Cross-Modality Rebalance
• GNCAF: A GNN-based Neighboring Context Aggregation Framework for Tertiary Lymphoid Structures Semantic Segmentation in WSI
• Self-Supervision Enhances Instance-based Multiple Instance Learning Methods in Digital Pathology: A Benchmark Study
• A Multi-Cohort Validation of Censoring-Aware Conformal Lower Predictive Bounds for Pathology Survival Models
• Predicting ulcer in H&E images of inflammatory bowel disease using domain-knowledge-driven graph neural network

Related

• Spatially-Aware Mixture of Experts with Log-Logistic Survival Modeling for Whole-Slide Images
• Survival Modeling from Whole Slide Images via Patch-Level Graph Clustering and Mixture Density Experts
• C2P-GCN: Cell-to-Patch Graph Convolutional Network for Colorectal Cancer Grading
• IPGPhormer: Interpretable Pathology Graph-Transformer for Survival Analysis
• Survival Prediction Across Diverse Cancer Types Using Neural Networks