Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models

2025/02/27 by Yukang Yang, Declan Campbell, Yang, Yukang +10 · 2 voices · 9 citations
Computer Science · Social Sciences · #Explainable Artificial Intelligence (XAI) #Language and cultural evolution #Topic Modeling #cs.AI #cs.CL

paper · pdf · doi:10.48550/arxiv.2502.20332

openalex publication_date 2025/02/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Many recent studies have found evidence for emergent reasoning capabilities in large language models (LLMs), but debate persists concerning the robustness of these capabilities, and the extent to which they depend on structured reasoning mechanisms. To shed light on these issues, we study the internal mechanisms that support abstract reasoning in LLMs. We identify an emergent symbolic architecture that implements abstract reasoning via a series of three computations. In early layers, symbol abstraction heads convert input tokens to abstract variables based on the relations between those tokens. In intermediate layers, symbolic induction heads perform sequence induction over these abstract variables. Finally, in later layers, retrieval heads predict the next token by retrieving the value associated with the predicted abstract variable. These results point toward a resolution of the longstanding debate between symbolic and neural network approaches, suggesting that emergent reasoning in neural networks depends on the emergence of symbolic mechanisms.

Citations

• Which Attention Heads Matter for In-Context Learning?
• Evaluating the Robustness of Analogical Reasoning in Large Language Models
• Evaluating Cognitive Maps and Planning in Large Language Models with CogEval
• Large Language Models as General Pattern Machines
• From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought
• On the Dangers of Stochastic Parrots
• Attention Is All You Need

Cited by

• Whither symbols in the era of advanced neural networks?
• In-Context Algebra
• In-Context Learning Without Copying
• LLMs Process Lists With General Filter Heads
• The Mechanistic Emergence of Symbol Grounding in Language Models
• Representing Beauty: Towards a Participatory but Objective Latent Aesthetics
• Learning to Ponder: Adaptive Reasoning in Latent Space
• A circuit for predicting hierarchical structure in-context in Large Language Models
• Decomposing Representation Space into Interpretable Subspaces with Unsupervised Learning

Discussions

• LLMs have shown impressive performance in some reasoning tasks, but what internal mechanisms do they use to solve these tasks? In a new preprint, we find evidence that abstract reasoning in LLMs depen [bsky, 118 points, 5 comments]
• arxiv.org/abs/2502.203... Maybe LLM's are using symbolic mechanisms built through transformers? Raw transformers are only the "machine code" explanation for how LLM's incredible performance. I really [bsky, 0 points, 0 comments]

Related

• Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners
• Neuro-Symbolic Artificial Intelligence: Towards Improving the Reasoning Abilities of Large Language Models
• Interpretable by AI Mother Tongue: Native Symbolic Reasoning in Neural Models
• Current Practices for Building LLM-Powered Reasoning Tools Are Ad Hoc -- and We Can Do Better
• Benchmarking Abstract and Reasoning Abilities Through A Theoretical Perspective