On the Semantics of Large Language Models

2025/07/07 by Martin Schuele, Schuele, Martin
Computer Science · Medicine · Social Sciences · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Healthcare and Education #Computation and Language (cs.CL) #Ethics and Social Impacts of AI #FOS: Computer and information sciences #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2507.05448

openalex publication_date 2025/07/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large Language Models (LLMs) such as ChatGPT demonstrated the potential to replicate human language abilities through technology, ranging from text generation to engaging in conversations. However, it remains controversial to what extent these systems truly understand language. We examine this issue by narrowing the question down to the semantics of LLMs at the word and sentence level. By examining the inner workings of LLMs and their generated representation of language and by drawing on classical semantic theories by Frege and Russell, we get a more nuanced picture of the potential semantic capabilities of LLMs.

Citations

• Mamba: Linear-Time Sequence Modeling with Selective State Spaces
• Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence
• A Survey on Evaluation of Large Language Models
• A Survey on Evaluation of Large Language Models
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model
• The Vector Grounding Problem
• Sparks of Artificial General Intelligence: Early experiments with GPT-4
• Break It Down: Evidence for Structural Compositionality in Neural Networks
• Discovering Latent Knowledge in Language Models Without Supervision
• Meaning without reference in large language models
• On the Dangers of Stochastic Parrots
• A Primer in BERTology: What we know about how BERT works
• Scaling Laws for Neural Language Models
• Deep reinforcement learning from human preferences
• Distributed Representations of Words and Phrases and their Compositionality
• Wittgenstein on Meaning and Use
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Related

• Do Language Models Have Semantics? On the Five Standard Positions
• Meaning and understanding in large language models
• Large Language Models Understanding: an Inherent Ambiguity Barrier
• The Quo Vadis of the Relationship between Language and Large Language Models
• Large Language Models: The Need for Nuance in Current Debates and a Pragmatic Perspective on Understanding