Large Language Models in Qualitative Research: Uses, Tensions, and Intentions

2024/10/09 by Hope Schroeder, Marianne Aubin Le Quéré, Schroeder, Hope +7 · 14 citations
Social Sciences · #Computational and Text Analysis Methods #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC)

paper · pdf · doi:10.48550/arxiv.2410.07362

openalex publication_date 2024/10/09 · openalex created_date 2024/10/13 · openalex updated_date 2026/07/28

Abstract

Qualitative researchers use tools to collect, sort, and analyze their data. Should qualitative researchers use large language models (LLMs) as part of their practice? LLMs could augment qualitative research, but it is unclear if their use is appropriate, ethical, or aligned with qualitative researchers' goals and values. We interviewed twenty qualitative researchers to investigate these tensions. Many participants see LLMs as promising interlocutors with attractive use cases across the stages of research, but wrestle with their performance and appropriateness. Participants surface concerns regarding the use of LLMs while protecting participant interests, and call attention to an urgent lack of norms and tooling to guide the ethical use of LLMs in research. We document the rapid and broad adoption of LLMs across surfaces, which can interfere with intentional use vital to qualitative research. We use the tensions surfaced by our participants to outline recommendations for researchers considering using LLMs in qualitative research and design principles for LLM-assisted qualitative research tools.

Cited by

• With Visual Integrity and Care: A Framework for Mixed Methods Research on Visual Social Data
• When LLMs fall short in Deductive Coding: Model Comparison and Human AI Collaboration Workflow Design
• Can machines perform a qualitative data analysis? Reading the debate with Alan Turing
• LLM-Assisted Thematic Analysis: Opportunities, Limitations, and Recommendations
• AI Annotation Orchestration: Evaluating LLM verifiers to Improve the Quality of LLM Annotations in Learning Analytics
• Will Scaling Improve Social Simulation with LLMs?
• DeTAILS: Deep Thematic Analysis with Iterative LLM Support
• Product Manager Practices for Delegating Work to Generative AI: "Accountability must not be delegated to non-human actors"
• Populism Meets AI: Advancing Populism Research with LLMs
• LOGOS: LLM-driven End-to-End Grounded Theory Development and Schema Induction for Qualitative Research
• Qualitative Research in an Era of AI: A Pragmatic Approach to Data Analysis, Workflow, and Computation
• Empowering Computing Education Researchers Through LLM-Assisted Content Analysis
• Prosocial Behavior Detection in Player Game Chat: From Aligning Human-AI Definitions to Efficient Annotation at Scale
• Show or Tell? Modeling the evolution of request-making in Human-LLM conversations

Related

• Uses and Misuses of Large Language Models in Qualitative Research
• 'Simulacrum of Stories': Examining Large Language Models as Qualitative Research Participants
• Large Language Model for Qualitative Research -- A Systematic Mapping Study
• A Framework For Discussing LLMs as Tools for Qualitative Analysis
• LLMs as Research Tools: A Large Scale Survey of Researchers' Usage and Perceptions