Public Perceptions of Gender Bias in Large Language Models: Cases of ChatGPT and Ernie

2023/09/17 by Kyrie Zhixuan Zhou, Madelyn Rose Sanfilippo, Zhou, Kyrie Zhixuan +1 · 1 citation
Social Sciences · #Artificial Intelligence (cs.AI) #Computers and Society (cs.CY) #FOS: Computer and information sciences #Gender Studies in Language

paper · pdf · doi:10.48550/arxiv.2309.09120

openalex publication_date 2023/09/17 · openalex created_date 2023/09/20 · openalex updated_date 2026/07/28

Abstract

Large language models are quickly gaining momentum, yet are found to demonstrate gender bias in their responses. In this paper, we conducted a content analysis of social media discussions to gauge public perceptions of gender bias in LLMs which are trained in different cultural contexts, i.e., ChatGPT, a US-based LLM, or Ernie, a China-based LLM. People shared both observations of gender bias in their personal use and scientific findings about gender bias in LLMs. A difference between the two LLMs was seen -- ChatGPT was more often found to carry implicit gender bias, e.g., associating men and women with different profession titles, while explicit gender bias was found in Ernie's responses, e.g., overly promoting women's pursuit of marriage over career. Based on the findings, we reflect on the impact of culture on gender bias and propose governance recommendations to regulate gender bias in LLMs.

Cited by

• An Empirical Investigation of Gender Stereotype Representation in Large Language Models: The Italian Case

Related

• Unveiling Gender Bias in Terms of Profession Across LLMs: Analyzing and Addressing Sociological Implications
• Gender bias and stereotypes in Large Language Models
• Assessing Gender Bias in LLMs: Comparing LLM Outputs with Human Perceptions and Official Statistics
• Gender Biases in LLMs: Higher intelligence in LLM does not necessarily solve gender bias and stereotyping
• "Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters