2024/03/18 by Hongxiao Wang, Wang, Hongxiao, Yang Yang +9 · 1 citation
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Biological sciences #FOS: Computer and information sciences #Genomics (q-bio.GN) #Machine Learning (cs.LG) #Machine Learning in Healthcare
paper · pdf · doi:10.48550/arxiv.2403.11375
openalex publication_date 2024/03/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
For predicting cancer survival outcomes, standard approaches in clinical research are often based on two main modalities: pathology images for observing cell morphology features, and genomic (e.g., bulk RNA-seq) for quantifying gene expressions. However, existing pathology-genomic multi-modal algorithms face significant challenges: (1) Valuable biological insights regarding genes and gene-gene interactions are frequently overlooked; (2) one modality often dominates the optimization process, causing inadequate training for the other modality. In this paper, we introduce a new multi-modal ``Path-GPTOmic" framework for cancer survival outcome prediction. First, to extract valuable biological insights, we regulate the embedding space of a foundation model, scGPT, initially trained on single-cell RNA-seq data, making it adaptable for bulk RNA-seq data. Second, to address the imbalance-between-modalities problem, we propose a gradient modulation mechanism tailored to the Cox partial likelihood loss for survival prediction. The contributions of the modalities are dynamically monitored and adjusted during the training process, encouraging that both modalities are sufficiently trained. Evaluated on two TCGA(The Cancer Genome Atlas) datasets, our model achieves substantially improved survival prediction accuracy.