Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers

2024/06/26 by Yibo Jiang, Goutham Rajendran, Jiang, Yibo +5 · 4 citations
Computer Science · #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.2406.18400

Abstract

Large Language Models (LLMs) have the capacity to store and recall facts. Through experimentation with open-source models, we observe that this ability to retrieve facts can be easily manipulated by changing contexts, even without altering their factual meanings. These findings highlight that LLMs might behave like an associative memory model where certain tokens in the contexts serve as clues to retrieving facts. We mathematically explore this property by studying how transformers, the building blocks of LLMs, can complete such memory tasks. We study a simple latent concept association problem with a one-layer transformer and we show theoretically and empirically that the transformer gathers information using self-attention and uses the value matrix for associative memory.

Cited by

• EnchTable: Unified Safety Alignment Transfer in Fine-tuned Large Language Models
• UniFormer: Unified and Efficient Transformer for Reasoning Across General and Custom Computing
• Understanding LoRA as Knowledge Memory: An Empirical Analysis
• Large Language Models as Model Organisms for Human Associative Learning

Related

• Understanding Factual Recall in Transformers via Associative Memories
• Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMs
• Birth of a Transformer: A Memory Viewpoint
• Understanding Transformer from the Perspective of Associative Memory
• Do All Autoregressive Transformers Remember Facts the Same Way? A Cross-Architecture Analysis of Recall Mechanisms