The debate over understanding in AI’s large language models

2022/10/14 by Melanie Mitchell, David C. Krakauer · 3 voices · 36 citations
Computer Science · Psychology · Social Sciences · #Language and cultural evolution #Language, Metaphor, and Cognition #Topic Modeling #cs.AI #cs.LG

paper · pdf · doi:10.1073/pnas.2215907120

openalex publication_date 2023/03/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29

Abstract

We survey a current, heated debate in the artificial intelligence (AI) research community on whether large pretrained language models can be said to understand language-and the physical and social situations language encodes-in any humanlike sense. We describe arguments that have been made for and against such understanding and key questions for the broader sciences of intelligence that have arisen in light of these arguments. We contend that an extended science of intelligence can be developed that will provide insight into distinct modes of understanding, their strengths and limitations, and the challenge of integrating diverse forms of cognition.

Citations

• No understanding without explanation
• SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
• Shortcut learning in deep neural networks
• On the Dangers of Stochastic Parrots
• Highly accurate protein structure prediction with AlphaFold
• Grounded Cognition
• Aion Framework: Dimensional Emergence of AI Consciousness, Observer-Induced Collapse, and Cosmological Portal Dynamics

Cited by

• Coherent without Grounding, Grounded without Success: The Bidirectional Coherence Paradox in Artificial Epistemic Agents
• Large language models are not about natural language
• Beyond Mimicry: Preference Coherence in LLMs
• Quantification and object perception in Multimodal Large Language Models and human linguistic cognition
• Use and usability: concepts of representation in philosophy, neuroscience, cognitive science, and computer science
• Large Language Model Agent Personality and Response Appropriateness: Evaluation by Human Linguistic Experts, LLM-as-Judge, and Natural Language Processing Model
• The end of experimental research as we know it? A perspective on generative artificial intelligence in communication science
• Readers Prefer Outputs of AI Trained on Copyrighted Books over Expert Human Writers
• Creating a large language model of a philosopher
• Assessing the nature of large language models: A caution against anthropocentrism
• Large Language Models can extract morphological data from taxonomic descriptions, but their stochastic nature makes automation challenging: a test on Australian Asteraceae
• Revealing emergent human-like conceptual representations from language prediction
• Extreme Self-Preference in Language Models
• Comparison of Large Language Model with Aphasia
• Exploring the scope of generative AI in literature review development
• The linguistic dead zone of value-aligned agency, natural and artificial
• When Can AI Models Explain Learning? Validity Criteria for AI as Cognitive Models in Education
• Psychometrically derived 60-question benchmarks: Substantial efficiencies and the possibility of human-AI comparisons
• The Philosophy of Language Models
• Artificial Intelligence’s new clothes? A system technology perspective
• Beyond computational equivalence: the behavioral inference principle for machine consciousness
• Are Language Models Models?
• A Matter of Time: Towards a General Theory of Agency
• Large language model-driven agents in nursing practice: A scoping review
• The Quantum Ecology as an Onto-Epistemological Framework: Toward an Ecological Theory of Sensing
• From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency
• Künstliche Intelligenz in den Naturwissenschaftsdidaktiken – gekommen, um zu bleiben: Potenziale, Desiderata, Herausforderungen
• Concepts
• The Turing test is not a good benchmark for thought in LLMs
• The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs
• Assessing Consciousness-Related Behaviors in Large Language Models Using the Maze Test
• Inducing State Anxiety in LLM Agents Reproduces Human-Like Biases in Consumer Decision-Making
• Humans Learn Language from Situated Communicative Interactions. What about Machines?
• The benefits and dangers of anthropomorphic conversational agents
• Intuition emerges in Maximum Caliber models at criticality
• Breaking the mould of Social Mixed Reality - State-of-the-Art and Glossary

Discussions

• Es gibt 2 Positionen in der KI-Forschung: 1. LLMs verfügen über eine Form von begrenztem internem Wissen. 2. LLMs sind "stochastische Papageien". Melanie Mitchell vertritt eine Zwischenposition: LLMs [bsky, 15 points, 3 comments]
• The Debate over Understanding in AI's Large Language Models [hn, 3 points, 0 comments]
• The Debate over Understanding in AI's Large Language Models[pdf] [hn, 2 points, 0 comments]

Related

• Understanding Natural Language Understanding Systems. A Critical Analysis
• Talking About Large Language Models
• How should the generative power of Large Language Models (LLMs) be interpreted? Do chatbots understand linguistic meaning?
• A Philosophical Introduction to Language Models -- Part I: Continuity With Classic Debates
• How important is language for human-like intelligence?