V1T: large-scale mouse V1 response prediction using a Vision Transformer

2023/02/06 by Li, Bryan M., Cornacchia, Isabel M., Rochefort, Nathalie L. +1 · 3 citations
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Biological sciences #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural and Evolutionary Computing (cs.NE) #Neurons and Cognition (q-bio.NC)

paper · doi:10.48550/arxiv.2302.03023

Abstract

Accurate predictive models of the visual cortex neural response to natural visual stimuli remain a challenge in computational neuroscience. In this work, we introduce V1T, a novel Vision Transformer based architecture that learns a shared visual and behavioral representation across animals. We evaluate our model on two large datasets recorded from mouse primary visual cortex and outperform previous convolution-based models by more than 12.7% in prediction performance. Moreover, we show that the self-attention weights learned by the Transformer correlate with the population receptive fields. Our model thus sets a new benchmark for neural response prediction and can be used jointly with behavioral and neural recordings to reveal meaningful characteristic features of the visual cortex.

Cited by

• Manifolds and Modules: How Function Develops in a Neural Foundation Model
• MEIcoder: Decoding Visual Stimuli from Neural Activity by Leveraging Most Exciting Inputs
• Learning to cluster neuronal function

Related

• The Dynamic Sensorium competition for predicting large-scale mouse visual cortex activity from videos
• The Sensorium competition on predicting large-scale mouse primary visual cortex activity
• Retrospective for the Dynamic Sensorium Competition for predicting large-scale mouse primary visual cortex activity from videos
• AVM: Towards Structure-Preserving Neural Response Modeling in the Visual Cortex Across Stimuli and Individuals
• Modeling Dynamic Neural Activity by combining Naturalistic Video Stimuli and Stimulus-independent Latent Factors