Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking

2024/09/24 by Harsh Kumar, Jonathan Vincentius, Ewan Jordan +1 · 14 voices · 19 citations
Psychology · Engineering · Business, Management and Accounting · #Creativity in Education and Neuroscience #Design Education and Practice #Entrepreneurship Studies and Influences

paper · pdf · doi:10.1145/3706598.3714198

Abstract

Large language models are transforming the creative process by offering unprecedented capabilities to algorithmically generate ideas. While these tools can enhance human creativity when people co-create with them, it's unclear how this will impact unassisted human creativity. We conducted two large pre-registered parallel experiments involving 1,100 participants attempting tasks targeting the two core components of creativity, divergent and convergent thinking. We compare the effects of two forms of large language model (LLM) assistance -- a standard LLM providing direct answers and a coach-like LLM offering guidance -- with a control group receiving no AI assistance, and focus particularly on how all groups perform in a final, unassisted stage. Our findings reveal that while LLM assistance can provide short-term boosts in creativity during assisted tasks, it may inadvertently hinder independent creative performance when users work without assistance, raising concerns about the long-term impact on human creativity and cognition.

Citations

• Use of large language models might affect our cognitive skills

Cited by

• Developing and Validating the Spanish Version of the Large Language Models Dependency Scale (LLM-D12-SP)
• Introducing AI to an Online Petition Platform Changed Outputs but not Outcomes
• Invisible Saboteurs: Sycophantic LLMs Mislead Novices in Problem-Solving Tasks
• Recognising, Anticipating, and Mitigating LLM Pollution of Online Behavioural Research
• The indiscriminate adoption of AI threatens the foundations of academia
• Exploration vs. Fixation: Scaffolding Divergent and Convergent Thinking for Human-AI Co-Creation with Generative Models
• Large Language Models as Search Engines: Societal Challenges
• AnimAgents: Coordinating Multi-Stage Animation Pre-Production with Human-Multi-Agent Collaboration
• Scaffolding Creativity: How Divergent and Convergent LLM Personas Shape Human Machine Creative Problem-Solving
• Partnering with Generative AI: Experimental Evaluation of Human-Led and Model-Led Interaction in Human-AI Co-Creation
• Galton's Law of Mediocrity: Why Large Language Models Regress to the Mean and Fail at Creativity in Advertising
• Recognising and mitigating LLM Pollution in online behavioural research
• FlexMind: Supporting Deeper Creative Thinking with LLMs
• Measuring Large Language Models Dependency: Validating the Arabic Version of the LLM-D12 Scale
• Vibe Coding for UX Design: Understanding UX Professionals' Perceptions of AI-Assisted Design and Development
• Understanding, Protecting, and Augmenting Human Cognition with Generative AI: A Synthesis of the CHI 2025 Tools for Thought Workshop
• GenTune: Toward Traceable Prompts to Improve Controllability of Image Refinement in Environment Design
• ConlangCrafter: Constructing Languages with a Multi-Hop LLM Pipeline
• CoGrader: Transforming Instructors' Assessment of Project Reports through Collaborative LLM Integration

Discussions

• Hi there, if you’re like - hey, LLMs those seem cool and useful! Well, they do weird things to your brain and in certain instances and can impact your creative thinking even AFTER you stop using them [bsky, 143 points, 8 comments]
• LLMs will rot your brain 🤪 (read the study or at least the abstract — it’s free) arxiv.org/abs/2410.03703 [bsky, 26 points, 2 comments]
• AND that the only way gen ai can exist is from mass theft of copyrighted materials. AND what it does to the brain’s ability to independently be creative after ai has been used in the process. This tal [bsky, 8 points, 1 comments]
• Spannend. arxiv.org/pdf/2410.03703 [bsky, 6 points, 3 comments]
• Large pre-registered experiments (1100 participants) compare LLM assistance with no AI assistance on creative tasks. Findings: LLM assistance can boost short-term creativity during assisted tasks, b [bsky, 5 points, 0 comments]
• presenting today, yet again, about the scientifically backed harmful impact of AI (starring this UoT study arxiv.org/pdf/2410.03703) being indiscriminately adopted across our org after our CTO called [bsky, 4 points, 0 comments]
• I posted about this a while back - and as I understand, it is still a preprint so requires a grain of salt - but there is emerging evidence that using GenAI can leave people less equipped to operate w [bsky, 2 points, 1 comments]
• It's AI usage arxiv.org/abs/2410.03703 Using it repeatedly reduces your ability to think, EVEN AFTER YOU STOP USING IT. [bsky, 2 points, 0 comments]
• Thought this might be an interesting read to anyone who has thought about using gen AI arxiv.org/abs/2410.03703 [bsky, 2 points, 0 comments]
• arxiv.org/abs/2410.03703 [bsky, 1 points, 0 comments]
• high chance next year or that that papers will appear saying ai makes you dumber bc we already have a paper saying ai makes you less creative! arxiv.org/abs/2410.03703 (keep in mind this is not pee [bsky, 1 points, 0 comments]
• This study of LLMs's effects on creativity suggests that "while LLM assistance can provide short-term boosts in creativity..., it may inadvertently hinder independent creative performance when users w [bsky, 1 points, 1 comments]
• Today in reading LLM research: there is some evidence that use of LLMs makes you come up with creative ideas similar to other people who use LLMs. Literally "I laugh because you're all the same." arxi [bsky, 1 points, 1 comments]
• Los "expertos" "innovadores" educativos de la IA al final sí van a conseguir matar la creatividad. arxiv.org/pdf/2410.03703 [bsky, 0 points, 0 comments]

Related

• Scaffolding Creativity: How Divergent and Convergent LLM Personas Shape Human Machine Creative Problem-Solving
• Large Language Models show both individual and collective creativity comparable to humans
• We're Different, We're the Same: Creative Homogeneity Across LLMs
• How and for whom using generative AI affects creativity: A field experiment.
• Assessing and Understanding Creativity in Large Language Models