2025/12/05 by Vahid Golderzahi, Guanjie Wang, Jiyuan Hu +1 · 1 voice
Neuroscience · Computer Science · Medicine · #Face Recognition and Perception #Explainable Artificial Intelligence (XAI) #Radiomics and Machine Learning in Medical Imaging
paper · doi:10.64898/2025.11.20.689376
openalex publication_date 2025/12/05 · openalex created_date 2025/12/08 · openalex updated_date 2026/07/31
Abstract Both genotype and imaging data carry entangled information about the phenotypes. Associations between genotypes and phenotypes can be manifested or absent on biomedical images. While there are abundant multimodal association studies integrating genotype and imaging data, few of them disentangle their direct and indirect effects in associations. We propose GIF (Genotype-Image-Phenotype), a novel modeling framework to predict phenotypes by genotype and imaging data with a mediation structure of these three sets of variables. GIF constitutes a backbone of mediation path genotype→image→phenotype, and direct genotype→phenotype and image→phenotype paths capturing the additional direct associations not explained by the mediation model. To implement the mediation model with a convolutional neural network (CNN), we constructed a CNN to predict genotypes with images and then employed the joint embedding vectors of the CNN to predict phenotypes. The direct links were sequentially augmented to the mediation model to further improve prediction accuracy. We validated GIF on three datasets: (1) a synthetic polygon dataset where the presence or absence of a polygon specie indicates a genotype and selected combinations of multiple polygon species indicate phenotypes, (2) the PASCAL VOL dataset of object detection and action recognition, where the presence or absence of an object class indicates a genotype and the presence or absence of an action class indicates a phenotype, (3) the ADNI dataset of Alzheimer’s disease diagnosis comprising the genotype and imaging data of subjects with three cognitive states. GIF prediction outcomes on the ADNI data indicate the associations between genotypes and phenotypes are primarily mediated by image features, and images provide additional information about phenotypes which is not attributed to genotype variations.