Multimodal Generalized Zero Shot Learning for Gleason Grading using Self-Supervised Learning

2021/11/15 by Dwarikanath Mahapatra, Mahapatra, Dwarikanath
Computer Science · Medicine · #AI in cancer detection #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #Radiomics and Machine Learning in Medical Imaging #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2111.07646

openalex publication_date 2021/11/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Gleason grading from histopathology images is essential for accurate prostate cancer (PCa) diagnosis. Since such images are obtained after invasive tissue resection quick diagnosis is challenging under the existing paradigm. We propose a method to predict Gleason grades from magnetic resonance (MR) images which are non-interventional and easily acquired. We solve the problem in a generalized zero-shot learning (GZSL) setting since we may not access training images of every disease grade. Synthetic MRI feature vectors of unseen grades (classes) are generated by exploiting Gleason grades' ordered nature through a conditional variational autoencoder (CVAE) incorporating self-supervised learning. Corresponding histopathology features are generated using cycle GANs, and combined with MR features to predict Gleason grades of test images. Experimental results show our method outperforms competing feature generating approaches for GZSL, and comes close to performance of fully supervised methods.

Citations

• Adam: A Method for Stochastic Optimization
• Consensus Based Medical Image Segmentation Using Semi-Supervised\n Learning And Graph Cuts
• Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
• Terabyte-scale Deep Multiple Instance Learning for Classification and\n Localization in Pathology
• Chest X-rays Classification: A Multi-Label and Fine-Grained Problem
• GAN Based Medical Image Registration
• Registration of presurgical MRI and histopathology images from radical prostatectomy via RAPSODI
• Anomaly Detection on X-Rays Using Self-Supervised Aggregation Learning

Related

• Weakly-Supervised Deep Learning Model for Prostate Cancer Diagnosis and Gleason Grading of Histopathology Images
• Gleason Score Prediction using Deep Learning in Tissue Microarray Image
• Classification of Gleason Grading in Prostate Cancer Histopathology Images Using Deep Learning Techniques: YOLO, Vision Transformers, and Vision Mamba
• Assessing the Performance of Deep Learning for Automated Gleason Grading in Prostate Cancer
• Automated Prostate Cancer Diagnosis Based on Gleason Grading Using Convolutional Neural Network