What's in a Name? Auditing Large Language Models for Race and Gender Bias

2024/02/21 by Alejandro Salinas, Salinas, Alejandro, Amit Haim +3 · 11 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computers and Society (cs.CY) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2402.14875

openalex publication_date 2024/02/21 · openalex created_date 2024/02/27 · openalex updated_date 2026/07/28

Abstract

We employ an audit design to investigate biases in state-of-the-art large language models, including GPT-4. In our study, we prompt the models for advice involving a named individual across a variety of scenarios, such as during car purchase negotiations or election outcome predictions. We find that the advice systematically disadvantages names that are commonly associated with racial minorities and women. Names associated with Black women receive the least advantageous outcomes. The biases are consistent across 42 prompt templates and several models, indicating a systemic issue rather than isolated incidents. While providing numerical, decision-relevant anchors in the prompt can successfully counteract the biases, qualitative details have inconsistent effects and may even increase disparities. Our findings underscore the importance of conducting audits at the point of LLM deployment and implementation to mitigate their potential for harm against marginalized communities.

Cited by

• Privacy Practices of Browser Agents
• Evaluating Implicit Biases in LLM Reasoning through Logic Grid Puzzles
• Marked Pedagogies: Examining Linguistic Biases in Personalized Automated Writing Feedback
• All You Need is One: Capsule Prompt Tuning with a Single Vector
• Investigating Political and Demographic Associations in Large Language Models Through Moral Foundations Theory
• AI for Abolition? A Participatory Design Approach
• From Superficial Outputs to Superficial Learning: Risks of Large Language Models in Education
• Clustering Discourses: Racial Biases in Short Stories about Women Generated by Large Language Models
• DAIQ: Auditing Demographic Attribute Inference from Question in LLMs
• Yet another algorithmic bias: A Discursive Analysis of Large Language Models Reinforcing Dominant Discourses on Gender and Race
• Fairness Is Not Enough: Auditing Competence and Intersectional Bias in AI-powered Resume Screening

Related

• Prompt and Prejudice
• Auditing the Use of Language Models to Guide Hiring Decisions
• Fine-Grained Bias Detection in LLM: Enhancing detection mechanisms for nuanced biases
• "You Gotta be a Doctor, Lin": An Investigation of Name-Based Bias of Large Language Models in Employment Recommendations
• No Free Lunch in Language Model Bias Mitigation? Targeted Bias Reduction Can Exacerbate Unmitigated LLM Biases