2023/06/03 by Han Yi Chiu, Liang Zhao, Chiu, Han Yi +3 · 1 citation
Computer Science · Neuroscience · #Advanced Graph Neural Networks #FOS: Biological sciences #FOS: Computer and information sciences #Functional Brain Connectivity Studies #Machine Learning (cs.LG) #Neural dynamics and brain function #Neurons and Cognition (q-bio.NC)
paper · pdf · doi:10.48550/arxiv.2306.05286
openalex publication_date 2023/06/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The decoding of brain neural networks has been an intriguing topic in neuroscience for a well-rounded understanding of different types of brain disorders and cognitive stimuli. Integrating different types of connectivity, e.g., Functional Connectivity (FC) and Structural Connectivity (SC), from multi-modal imaging techniques can take their complementary information into account and therefore have the potential to get better decoding capability. However, traditional approaches for integrating FC and SC overlook the dynamical variations, which stand a great chance to over-generalize the brain neural network. In this paper, we propose a Joint kernel Graph Attention Network (JGAT), which is a new multi-modal temporal graph attention network framework. It integrates the data from functional Magnetic Resonance Images (fMRI) and Diffusion Weighted Imaging (DWI) while preserving the dynamic information at the same time. We conduct brain-decoding tasks with our JGAT on four independent datasets: three of 7T fMRI datasets from the Human Connectome Project (HCP) and one from animal neural recordings. Furthermore, with Attention Scores (AS) and Frame Scores (FS) computed and learned from the model, we can locate several informative temporal segments and build meaningful dynamical pathways along the temporal domain for the HCP datasets. The URL to the code of JGAT model: https://github.com/BRAINML-GT/JGAT.