2024/05/10 by Jianyu Zhang, Zhang, Jianyu, Niklas Nolte +8 · 3 voices · 3 citations
Computer Science · Psychology · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Memory, Trauma, and Commemoration #Neural and Evolutionary Computing (cs.NE) #cs.AI #cs.LG #cs.NE
paper · pdf · doi:10.48550/arxiv.2405.06394
openalex publication_date 2024/05/10 · arxiv published 2024/05/10 · openalex created_date 2024/05/14 · arxiv updated 2025/02/27 · openalex updated_date 2026/07/28
Memory Mosaics are networks of associative memories working in concert to achieve a prediction task of interest. Like transformers, memory mosaics possess compositional capabilities and in-context learning capabilities. Unlike transformers, memory mosaics achieve these capabilities in comparatively transparent way ("predictive disentanglement"). We illustrate these capabilities on a toy example and also show that memory mosaics perform as well or better than transformers on medium-scale language modeling tasks.