Minimization of Boolean Complexity in In-Context Concept Learning

2024/12/03 by Leroy Z. Wang, R. Thomas McCoy, Wang, Leroy Z. +3 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computational Drug Discovery Methods #FOS: Computer and information sciences #Machine Learning and Algorithms

paper · pdf · doi:10.48550/arxiv.2412.02823

openalex publication_date 2024/12/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

What factors contribute to the relative success and corresponding difficulties of in-context learning for Large Language Models (LLMs)? Drawing on insights from the literature on human concept learning, we test LLMs on carefully designed concept learning tasks, and show that task performance highly correlates with the Boolean complexity of the concept. This suggests that in-context learning exhibits a learning bias for simplicity in a way similar to humans.

Cited by

• Uncovering Implicit Bias in Large Language Models with Concept Learning Dataset

Related

• Does In-Context Learning Really Learn? Rethinking How Large Language Models Respond and Solve Tasks via In-Context Learning
• Uncovering Implicit Bias in Large Language Models with Concept Learning Dataset
• Concept-aware Training Improves In-context Learning Ability of Language Models
• What Do Language Models Learn in Context? The Structured Task Hypothesis
• Concept-aware Data Construction Improves In-context Learning of Language Models