Pre-trained Large Language Models Use Fourier Features to Compute Addition

2024/06/05 by Tianyi Zhou, Deqing Fu, Zhou, Tianyi +5 · 11 voices · 10 citations
#cs.LG #cs.CL

paper · pdf · doi:10.48550/arxiv.2406.03445

Abstract

Pre-trained large language models (LLMs) exhibit impressive mathematical reasoning capabilities, yet how they compute basic arithmetic, such as addition, remains unclear. This paper shows that pre-trained LLMs add numbers using Fourier features -- dimensions in the hidden state that represent numbers via a set of features sparse in the frequency domain. Within the model, MLP and attention layers use Fourier features in complementary ways: MLP layers primarily approximate the magnitude of the answer using low-frequency features, while attention layers primarily perform modular addition (e.g., computing whether the answer is even or odd) using high-frequency features. Pre-training is crucial for this mechanism: models trained from scratch to add numbers only exploit low-frequency features, leading to lower accuracy. Introducing pre-trained token embeddings to a randomly initialized model rescues its performance. Overall, our analysis demonstrates that appropriate pre-trained representations (e.g., Fourier features) can unlock the ability of Transformers to learn precise mechanisms for algorithmic tasks.

Cited by

• Convergent Evolution: How Different Language Models Learn Similar Number Representations
• When Models Manipulate Manifolds: The Geometry of a Counting Task
• In-Context Algebra
• Sequential Enumeration in Large Language Models
• Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention
• Unravelling the Mechanisms of Manipulating Numbers in Language Models
• StreetMath: Study of LLMs' Approximation Behaviors
• Language Models Do Not Embed Numbers Continuously
• Reverse Browser: Vector-Image-to-Code Generator
• Modular Arithmetic: Language Models Solve Math Digit by Digit

Discussions

• Pre-Trained Large Language Models Use Fourier Features for Addition (2024) [hn, 149 points, 40 comments]
• we have a very much similar results in NeurIPS 2024: arxiv.org/abs/2406.03445 [bsky, 2 points, 1 comments]
• There are many humans who are worse at arithmetic than a state of the art LLM. They're many orders of magnitude less efficient than CPU arithmetic instructions but they can do it and we even know how: [bsky, 1 points, 2 comments]
• Pre-Trained Large Language Models Use Fourier Features for Addition (2024) https://arxiv.org/abs/2406.03445 https://news.ycombinator.com/item?id=42960989 [bsky, 0 points, 0 comments]
• Pre-Trained Large Language Models Use Fourier Features for Addition (2024) https://arxiv.org/abs/2406.03445 [bsky, 0 points, 0 comments]
• Pre-Trained Large Language Models Use Fourier Features for Addition (2024) https://arxiv.org/abs/2406.03445 [comments] [17 points] [bsky, 0 points, 0 comments]
• Pre-Trained Large Language Models Use Fourier Features for Addition (2024) https://arxiv.org/abs/2406.03445 (https://news.ycombinator.com/item?id=42960989) [bsky, 0 points, 0 comments]
• Pre-Trained Large Language Models Use Fourier Features for Addition (2024) (arxiv.org) Main Link | Discussion [bsky, 0 points, 0 comments]
• Links & presentation times: 1. Fourier Features: arxiv.org/abs/2406.03445 Thu, 4:30pm 2. TF + ICL: arxiv.org/abs/2310.17086 Fri, 11am 3. Backdoor detection: arxiv.org/abs/2409.00399 Sat, 1:44pm at Adv [bsky, 0 points, 0 comments]
• Pre-Trained Large Language Models Use Fourier Features for Addition (2024) https://arxiv.org/abs/2406.03445 (https://news.ycombinator.com/item?id=42960989) [bsky, 0 points, 0 comments]
• Pre-Trained Large Language Models Use Fourier Features for Addition (2024) [bsky, 0 points, 0 comments]

Related

• FoNE: Precise Single-Token Number Embeddings via Fourier Features
• Interpreting and Improving Large Language Models in Arithmetic Calculation
• Addition in Four Movements: Mapping Layer-wise Information Trajectories in LLMs
• Convergent Evolution: How Different Language Models Learn Similar Number Representations
• Language Models Use Trigonometry to Do Addition