The debate over understanding in AI’s large language models

2022/10/14 by Melanie Mitchell, David C. Krakauer · 3 voices · 62 citations
Computer Science · Psychology · Social Sciences · #Language and cultural evolution #Language, Metaphor, and Cognition #Topic Modeling #cs.AI #cs.LG

paper · pdf · doi:10.1073/pnas.2215907120

openalex publication_date 2023/03/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29

Abstract

We survey a current, heated debate in the artificial intelligence (AI) research community on whether large pretrained language models can be said to understand language-and the physical and social situations language encodes-in any humanlike sense. We describe arguments that have been made for and against such understanding and key questions for the broader sciences of intelligence that have arisen in light of these arguments. We contend that an extended science of intelligence can be developed that will provide insight into distinct modes of understanding, their strengths and limitations, and the challenge of integrating diverse forms of cognition.

Citations

• No understanding without explanation
• SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
• Shortcut learning in deep neural networks
• On the Dangers of Stochastic Parrots
• Highly accurate protein structure prediction with AlphaFold
• Grounded Cognition
• Aion Framework: Dimensional Emergence of AI Consciousness, Observer-Induced Collapse, and Cosmological Portal Dynamics

Cited by

• Coherent without Grounding, Grounded without Success: The Bidirectional Coherence Paradox in Artificial Epistemic Agents
• Large language models are not about natural language
• Beyond Mimicry: Preference Coherence in LLMs
• Quantification and object perception in Multimodal Large Language Models and human linguistic cognition
• Use and usability: concepts of representation in philosophy, neuroscience, cognitive science, and computer science
• Large Language Model Agent Personality and Response Appropriateness: Evaluation by Human Linguistic Experts, LLM-as-Judge, and Natural Language Processing Model
• The end of experimental research as we know it? A perspective on generative artificial intelligence in communication science
• Readers Prefer Outputs of AI Trained on Copyrighted Books over Expert Human Writers
• Creating a large language model of a philosopher
• Assessing the nature of large language models: A caution against anthropocentrism
• Large Language Models can extract morphological data from taxonomic descriptions, but their stochastic nature makes automation challenging: a test on Australian Asteraceae
• Revealing emergent human-like conceptual representations from language prediction
• Extreme Self-Preference in Language Models
• Comparison of Large Language Model with Aphasia
• Exploring the scope of generative AI in literature review development
• The linguistic dead zone of value-aligned agency, natural and artificial
• When Can AI Models Explain Learning? Validity Criteria for AI as Cognitive Models in Education
• Psychometrically derived 60-question benchmarks: Substantial efficiencies and the possibility of human-AI comparisons
• The Philosophy of Language Models
• Artificial Intelligence’s new clothes? A system technology perspective
• Beyond computational equivalence: the behavioral inference principle for machine consciousness
• Are Language Models Models?
• A Matter of Time: Towards a General Theory of Agency
• Large language model-driven agents in nursing practice: A scoping review
• The Quantum Ecology as an Onto-Epistemological Framework: Toward an Ecological Theory of Sensing
• From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency
• Künstliche Intelligenz in den Naturwissenschaftsdidaktiken – gekommen, um zu bleiben: Potenziale, Desiderata, Herausforderungen
• Concepts
• The Turing test is not a good benchmark for thought in LLMs
• The Personality Illusion: Revealing Dissociation Between Self-Reports & Behavior in LLMs
• Assessing Consciousness-Related Behaviors in Large Language Models Using the Maze Test
• Inducing State Anxiety in LLM Agents Reproduces Human-Like Biases in Consumer Decision-Making
• Humans Learn Language from Situated Communicative Interactions. What about Machines?
• The benefits and dangers of anthropomorphic conversational agents
• Intuition emerges in Maximum Caliber models at criticality
• Machine culture
• Breaking the mould of Social Mixed Reality - State-of-the-Art and Glossary
• Legal infrastructure for transformative AI governance
• Cognitive Network Science Reveals Bias in GPT-3, GPT-3.5 Turbo, and GPT-4 Mirroring Math Anxiety in High-School Students
• What if Othello-Playing Language Models Could See?
• Patterns, Models, and Challenges in Online Social Media: A Survey
• Model-Grounded Symbolic Artificial Intelligence Systems Learning and Reasoning with Model-Grounded Symbolic Artificial Intelligence Systems
• Cross-modal Associations in Vision and Language Models: Revisiting the Bouba-Kiki Effect
• Mechanistic Indicators of Understanding in Large Language Models
• Large Language Models for Zero-Shot Multicultural Name Recognition
• The role of LLMs in theory building
• Can structural correspondences ground real world representational content in Large Language Models?
• RE-IMAGINE: Symbolic Benchmark Synthesis for Reasoning Evaluation
• How large language models can reshape collective intelligence
• Revolutionizing Radiology Workflow with Factual and Efficient CXR Report Generation
• Neither Stochastic Parroting nor AGI: LLMs Solve Tasks through Context-Directed Extrapolation from Training Data Priors
• On the Same Page: Dimensions of Perceived Shared Understanding in Human-AI Interaction
• Language Model Behavior: A Comprehensive Survey
• Artificial intelligence and illusions of understanding in scientific research
• OntoURL: A Benchmark for Evaluating Large Language Models on Symbolic Ontological Understanding, Reasoning and Learning
• Context parroting: A simple but tough-to-beat baseline for foundation models in scientific machine learning
• Evaluating GPT- and Reasoning-based Large Language Models on Physics Olympiad Problems: Surpassing Human Performance and Implications for Educational Assessment
• Large Language Models Understanding: an Inherent Ambiguity Barrier
• A logical re-conception of neural networks: Hamiltonian bitwise part-whole architecture
• Alignment among Language, Vision and Action Representations
• Why AI Alignment Failure Is Structural: Learned Human Interaction Structures and AGI as an Endogenous Evolutionary Shock
• Credible Plan-Driven RAG Method for Multi-Hop Question Answering

Discussions

• Es gibt 2 Positionen in der KI-Forschung: 1. LLMs verfügen über eine Form von begrenztem internem Wissen. 2. LLMs sind "stochastische Papageien". Melanie Mitchell vertritt eine Zwischenposition: LLMs [bsky, 15 points, 3 comments]
• The Debate over Understanding in AI's Large Language Models [hn, 3 points, 0 comments]
• The Debate over Understanding in AI's Large Language Models[pdf] [hn, 2 points, 0 comments]

Related

• Understanding Natural Language Understanding Systems. A Critical Analysis
• Talking About Large Language Models
• How should the generative power of Large Language Models (LLMs) be interpreted? Do chatbots understand linguistic meaning?
• A Philosophical Introduction to Language Models -- Part I: Continuity With Classic Debates
• How important is language for human-like intelligence?