2025/08/02 by Raluca Rilla, Tobias Werner, Rilla, Raluca +8 · 2 voices · 4 citations
Medicine · Psychology · #Artificial Intelligence in Healthcare and Education #Compromise #Delegation #Digital Mental Health Interventions #Mediation #Mental Health via Writing #Premise #Spillover effect #Task (project management) #Vulnerability (computing)
paper · pdf · doi:10.48550/arxiv.2508.01390
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2025/08/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05
Online behavioural research faces an emerging threat as participants increasingly turn to large language models (LLMs) for advice, translation, or task delegation: LLM Pollution. We identify three interacting variants through which LLM Pollution threatens the validity and integrity of online behavioural research. First, Partial LLM Mediation occurs when participants make selective use of LLMs for specific aspects of a task, such as translation or wording support, leading researchers to (mis)interpret LLM-shaped outputs as human ones. Second, Full LLM Delegation arises when agentic LLMs complete studies with little to no human oversight, undermining the central premise of human-subject research at a more foundational level. Third, LLM Spillover signifies human participants altering their behaviour as they begin to anticipate LLM presence in online studies, even when none are involved. While Partial Mediation and Full Delegation form a continuum of increasing automation, LLM Spillover reflects second-order reactivity effects. Together, these variants interact and generate cascading distortions that compromise sample authenticity, introduce biases that are difficult to detect post hoc, and ultimately undermine the epistemic grounding of online research on human cognition and behaviour. Crucially, the threat of LLM Pollution is already co-evolving with advances in generative AI, creating an escalating methodological arms race. To address this, we propose a multi-layered response spanning researcher practices, platform accountability, and community efforts. As the challenge evolves, coordinated adaptation will be essential to safeguard methodological integrity and preserve the validity of online behavioural research.