Whither symbols in the era of advanced neural networks?

2025/08/07 by Thomas L. Griffiths, Brenden M. Lake, Griffiths, Thomas L. +7 · 4 voices · 2 citations
#cs.AI

paper · pdf · doi:10.48550/arxiv.2508.05776

Abstract

Some of the strongest evidence that human minds should be thought about in terms of symbolic systems has been the way they combine ideas, produce novelty, and learn quickly. We argue that modern neural networks -- and the artificial intelligence systems built upon them -- exhibit similar abilities. This undermines the argument that the cognitive processes and representations used by human minds are symbolic, although the fact that these neural networks are typically trained on data generated by symbolic systems illustrates that such systems play an important role in characterizing the abstract problems that human minds have to solve. This argument leads us to offer a new agenda for research on the symbolic basis of human thought.

Citations

• Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models
• Best-of-N Jailbreaking
• Centaur: a foundation model of human cognition
• Overcoming classic challenges for artificial neural networks by providing incentives and practice
• H-ARC: A Robust Estimate of Human Performance on the Abstraction and Reasoning Corpus Benchmark
• A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
• The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"
• Sparks of Artificial General Intelligence: Early experiments with GPT-4
• Training Compute-Optimal Large Language Models
• Thinking Like Transformers
• Language Models are Few-Shot Learners
• Attention Is All You Need
• Learning to reinforcement learn
• Neural Turing Machines
• The perceptron: A probabilistic model for information storage and organization in the brain.

Cited by

• Path Integration and Object-Location Binding Emerge in an Action-Conditioned Predictive Sequence Network
• How Do Language Models Compose Functions?

Discussions

• 🤖 🧠 NEW PAPER ON COGSCI & AI 🧠 🤖 Recent neural networks capture properties long thought to require symbols: compositionality, productivity, rapid learning So what role should symbols play in theor [bsky, 101 points, 8 comments]
• Interesting paper (read admittedly a little quickly, but not as long as it looks). Idk why this particular paper is making me think this, but nobody ever argues that Cicero is a model of human theory [bsky, 9 points, 2 comments]
• I am down a rabbit hole, so I am bookmarking these for myself: arxiv.org/abs/2508.05776 and transformer-circuits.pub/2022/in-cont... [bsky, 0 points, 0 comments]
• I like this general point about levels of explanation (which I think is similar to the point we make here arxiv.org/abs/2508.05776) but how does it relate to the discussion about mistakes made by LLMs [bsky, 0 points, 0 comments]

Related

• The Roles of Symbols in Neural-based AI: They are Not What You Think!
• Symbolic Behaviour in Artificial Intelligence
• Symbols as a Lingua Franca for Bridging Human-AI Chasm for Explainable and Advisable AI Systems
• Emergence of Symbols in Neural Networks for Semantic Understanding and Communication
• A survey on neural-symbolic learning systems