Are generative AI text annotations systematically biased?

2025/12/09 by Stolwijk, Sjoerd B., Boukes, Mark, Trilling, Damian
Computer Science · Medicine · Social Sciences · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Healthcare and Education #Computation and Language (cs.CL) #Computational and Text Analysis Methods #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2512.08404

openalex publication_date 2025/12/09 · openalex created_date 2025/12/11 · openalex updated_date 2026/07/28

Abstract

This paper investigates bias in GLLM annotations by conceptually replicating manual annotations of Boukes (2024). Using various GLLMs (Llama3.1:8b, Llama3.3:70b, GPT4o, Qwen2.5:72b) in combination with five different prompts for five concepts (political content, interactivity, rationality, incivility, and ideology). We find GLLMs perform adequate in terms of F1 scores, but differ from manual annotations in terms of prevalence, yield substantively different downstream results, and display systematic bias in that they overlap more with each other than with manual annotations. Differences in F1 scores fail to account for the degree of bias.

Citations

• Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation
• Large Language Models Outperform Expert Coders and Supervised Classifiers at Annotating Political Social Media Messages
• Surprising gender biases in GPT
• More human than human: measuring ChatGPT political bias
• Challenges and Applications of Large Language Models
• ChatGPT outperforms crowd workers for text-annotation tasks
• Prediction-Powered Inference
• Do Prompt-Based Models Really Understand the Meaning of their Prompts?

Related

• Keeping Humans in the Loop: Human-Centered Automated Annotation with Generative AI
• Evaluating how LLM annotations represent diverse views on contentious topics
• Automated Annotation with Generative AI Requires Validation
• Hey GPT, Can You be More Racist? Analysis from Crowdsourced Attempts to Elicit Biased Content from Generative AI
• GPT models for text annotation: An empirical exploration in public policy research