On Semantic Cognition, Inductive Generalization, and Language Models

2021/11/04 by Kanishka Misra, Misra, Kanishka
Computer Science · Social Sciences · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Language and cultural evolution #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2111.02603

openalex publication_date 2021/11/04 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

My doctoral research focuses on understanding semantic knowledge in neural network models trained solely to predict natural language (referred to as language models, or LMs), by drawing on insights from the study of concepts and categories grounded in cognitive science. I propose a framework inspired by 'inductive reasoning,' a phenomenon that sheds light on how humans utilize background knowledge to make inductive leaps and generalize from new pieces of information about concepts and their properties. Drawing from experiments that study inductive reasoning, I propose to analyze semantic inductive generalization in LMs using phenomena observed in human-induction literature, investigate inductive behavior on tasks such as implicit reasoning and emergent feature recognition, and analyze and relate induction dynamics to the learned conceptual representation space.

Related

• What Can Language Models Tell Us About Human Cognition?
• A Property Induction Framework for Neural Language Models
• Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement
• Injecting structural hints: Using language models to study inductive biases in language learning
• "Understanding AI": Semantic Grounding in Large Language Models