A Comparison of Large Language Model and Human Performance on Random Number Generation Tasks

2024/08/19 by Rachel M. Harrison, Harrison, Rachel M. · 1 voice · 3 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · #Advanced Text Analysis Techniques #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Biological sciences #FOS: Computer and information sciences #Neurons and Cognition (q-bio.NC) #Topic Modeling #cs.AI #cs.CL #q-bio.NC

paper · pdf · doi:10.48550/arxiv.2408.09656

openalex publication_date 2024/08/19 · arxiv published 2024/08/19 · arxiv updated 2024/08/20 · openalex created_date 2024/10/01 · openalex updated_date 2026/07/28

Abstract

Random Number Generation Tasks (RNGTs) are used in psychology for examining how humans generate sequences devoid of predictable patterns. By adapting an existing human RNGT for an LLM-compatible environment, this preliminary study tests whether ChatGPT-3.5, a large language model (LLM) trained on human-generated text, exhibits human-like cognitive biases when generating random number sequences. Initial findings indicate that ChatGPT-3.5 more effectively avoids repetitive and sequential patterns compared to humans, with notably lower repeat frequencies and adjacent number frequencies. Continued research into different models, parameters, and prompting methodologies will deepen our understanding of how LLMs can more closely mimic human random generation behaviors, while also broadening their applications in cognitive and behavioral science research.

Cited by

• In Search of the Ingredients of Open-Endedness: Replicating Picbreeder with Large Vision-Language Models
• One Token Is Enough: Fingerprinting and Verifying Large Language Models from Single-Token Output Distributions
• The Illusion of Stochasticity in LLMs

Discussions

• A Comparison of LLM and Human Performance on Random Number Generation Tasks[pdf] [hn, 1 points, 0 comments]

Related

• Deterministic or probabilistic? The psychology of LLMs as random number generators
• Towards a Psychology of Machines: Large Language Models Predict Human Memory
• Psychometric Predictive Power of Large Language Models
• Can LLMs Simulate Human Behavioral Variability? A Case Study in the Phonemic Fluency Task
• In-Context Learning Dynamics with Random Binary Sequences