Memory Encoding Model

2023/08/02 by Huzheng Yang, Yang, Huzheng, James C. Gee +3 · 1 citation
Engineering · Neuroscience · #Advanced Memory and Neural Computing #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Memory and Neural Mechanisms #Neural dynamics and brain function

paper · pdf · doi:10.48550/arxiv.2308.01175

openalex publication_date 2023/08/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We explore a new class of brain encoding model by adding memory-related information as input. Memory is an essential brain mechanism that works alongside visual stimuli. During a vision-memory cognitive task, we found the non-visual brain is largely predictable using previously seen images. Our Memory Encoding Model (Mem) won the Algonauts 2023 visual brain competition even without model ensemble (single model score 66.8, ensemble score 70.8). Our ensemble model without memory input (61.4) can also stand a 3rd place. Furthermore, we observe periodic delayed brain response correlated to 6th-7th prior image, and hippocampus also showed correlated activity timed with this periodicity. We conjuncture that the periodic replay could be related to memory mechanism to enhance the working memory.

Cited by

• SynBrain: Enhancing Visual-to-fMRI Synthesis via Probabilistic Representation Learning

Related

• Neural Encoding for Image Recall: Human-Like Memory
• BRAIN: Bias-Mitigation Continual Learning Approach to Vision-Brain Understanding
• Neuronal signatures of successful one-shot memory in mid-level visual cortex
• Brain and eye movement dynamics track the transition from learning to memory-guided action
• From seeing to remembering: Images with harder-to-reconstruct representations leave stronger memory traces