2025/04/03 by Joshua Holstein, Holstein, Joshua, Diener, Moritz +2 · 2 citations
Decision Sciences · Psychology · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Human-Automation Interaction and Safety #Human-Computer Interaction (cs.HC) #Personal Information Management and User Behavior
paper · pdf · doi:10.48550/arxiv.2504.02780
openalex publication_date 2025/04/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The rise of Generative AI, and Large Language Models (LLMs) in particular, is fundamentally changing cognitive processes in knowledge work, raising critical questions about their impact on human reasoning and problem-solving capabilities. As these AI systems become increasingly integrated into workflows, they offer unprecedented opportunities for augmenting human thinking while simultaneously risking cognitive erosion through passive consumption of generated answers. This tension is particularly pronounced in open-ended tasks, where effective solutions require deep contextualization and integration of domain knowledge. Unlike structured tasks with established metrics, measuring the quality of human-LLM interaction in such open-ended tasks poses significant challenges due to the absence of ground truth and the iterative nature of solution development. To address this, we present a framework that analyzes interaction patterns along two dimensions: cognitive activity mode (exploration vs. exploitation) and cognitive engagement mode (constructive vs. detrimental). This framework provides systematic measurements to evaluate when LLMs are effective tools for thought rather than substitutes for human cognition, advancing theoretical understanding and practical guidance for developing AI systems that protect and augment human cognitive capabilities.