The Illusion of Artificial Inclusion

2024/01/16 by William S. Agnew, William Agnew, A. Stevie Bergman +7 · 3 voices · 4 citations
Computer Science · Social Sciences · #Ethics and Social Impacts of AI #Innovative Human-Technology Interaction #Persona Design and Applications #cs.CY

paper · pdf · doi:10.1145/3613904.3642703

arxiv published 2024/01/16 · arxiv updated 2024/02/05 · openalex publication_date 2024/05/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/30

Abstract

Human participants play a central role in the development of modern artificial intelligence (AI) technology, in psychological science, and in user research. Recent advances in generative AI have attracted growing interest to the possibility of replacing human participants in these domains with AI surrogates. We survey several such “substitution proposals” to better understand the arguments for and against substituting human participants with modern generative AI. Our scoping review indicates that the recent wave of these proposals is motivated by goals such as reducing the costs of research and development work and increasing the diversity of collected data. However, these proposals ignore and ultimately conflict with foundational values of work with human participants: representation, inclusion, and understanding. This paper critically examines the principles and goals underlying human participation to help chart out paths for future work that truly centers and empowers participants.

Citations

• Using thematic analysis in psychology
• I.—COMPUTING MACHINERY AND INTELLIGENCE
• Racism and Research: The Case of the Tuskegee Syphilis Study
• Beyond Equity as Inclusion: A Framework of “Rightful Presence” for Guiding Justice-Oriented Studies in Teaching and Learning
• On the Dangers of Stochastic Parrots
• Flamingo: a Visual Language Model for Few-Shot Learning
• Aion Framework: Dimensional Emergence of AI Consciousness, Observer-Induced Collapse, and Cosmological Portal Dynamics
• Power to the People? Opportunities and Challenges for Participatory AI
• Survey of Hallucination in Natural Language Generation
• ChatGPT outperforms crowd workers for text-annotation tasks

Cited by

• Transphobia Is in the Eye of the Prompter: Trans-Centered Perspectives on Large Language Models
• Adding LLMs to the psycholinguistic norming toolbox: A practical guide to getting the most out of human ratings
• What Would an LLM Do? Evaluating Policymaking Capabilities of Large Language Models
• Recommending With, Not For: Co-Designing Recommender Systems for Social Good

Discussions

• Here is a review looking at the various things people are proposing (h/t @stephnie.bsky.social) Cutting data collection costs is a large motivator. [bsky, 4 points, 1 comments]
• This is the closest I’ve found: arxiv.org/abs/2401.08572 [bsky, 3 points, 1 comments]
• It's for UX research. So if the AI can find its way around a program, a human probably can as well. It's not for life and death science and hopefully no one uses it that way. arxiv.org/pdf/2401.085... [bsky, 2 points, 1 comments]

Related

• Provocation: Who benefits from "inclusion" in Generative AI?
• Reflections and Recommendations on AI Adoption Practice from a Mixed-Ability Research Group
• Simulating Subjects: The Promise and Peril of Artificial Intelligence Stand-Ins for Social Agents and Interactions
• A Vision for Operationalising Diversity and Inclusion in AI
• Power to the People? Opportunities and Challenges for Participatory AI