Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers

2024/06/26 by Yibo Jiang, Goutham Rajendran, Jiang, Yibo +5 · 1 voice · 6 citations
Computer Science · Engineering · Mathematics · Neuroscience · Psychology · #Artificial intelligence #Association (psychology) #Associative property #Bidirectional associative memory #Computer science #Content-addressable memory #Dream #Electrical engineering #Engineering #Mathematics #Neural Networks and Applications #Neuroscience #Psychology #Psychotherapist #Pure mathematics #Transformer #cs.CL #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2406.18400

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2024/06/26 · arxiv published 2024/06/26 · openalex created_date 2024/06/28 · arxiv updated 2024/11/27 · openalex updated_date 2026/08/01

Abstract

Large Language Models (LLMs) have the capacity to store and recall facts. Through experimentation with open-source models, we observe that this ability to retrieve facts can be easily manipulated by changing contexts, even without altering their factual meanings. These findings highlight that LLMs might behave like an associative memory model where certain tokens in the contexts serve as clues to retrieving facts. We mathematically explore this property by studying how transformers, the building blocks of LLMs, can complete such memory tasks. We study a simple latent concept association problem with a one-layer transformer and we show theoretically and empirically that the transformer gathers information using self-attention and uses the value matrix for associative memory.

Cited by

• EnchTable: Unified Safety Alignment Transfer in Fine-tuned Large Language Models
• UniFormer: Unified and Efficient Transformer for Reasoning Across General and Custom Computing
• Understanding LoRA as Knowledge Memory: An Empirical Analysis
• Large Language Models as Model Organisms for Human Associative Learning
• Pre-trained Large Language Models Learn Hidden Markov Models In-context
• Strong Memory, Weak Control: An Empirical Study of Executive Functioning in LLMs

Discussions

• I'm at #NeurIPS2024 this week to present my paper on fact retrieval (arxiv.org/abs/2406.18400, Wed 11 Dec 9 a.m.) and would love to connect to discuss representation and interpretability. Some of my f [bsky, 0 points, 0 comments]

Related

• Understanding Factual Recall in Transformers via Associative Memories
• Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMs
• Birth of a Transformer: A Memory Viewpoint
• Understanding Transformer from the Perspective of Associative Memory
• Do All Autoregressive Transformers Remember Facts the Same Way? A Cross-Architecture Analysis of Recall Mechanisms