Using Artificial French Data to Understand the Emergence of Gender Bias in Transformer Language Models

2023/10/24 by Lina Conti, Conti, Lina, Guillaume Wisniewski +1 · 2 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2310.15852

openalex publication_date 2023/10/24 · openalex created_date 2023/10/26 · openalex updated_date 2026/07/28

Abstract

Numerous studies have demonstrated the ability of neural language models to learn various linguistic properties without direct supervision. This work takes an initial step towards exploring the less researched topic of how neural models discover linguistic properties of words, such as gender, as well as the rules governing their usage. We propose to use an artificial corpus generated by a PCFG based on French to precisely control the gender distribution in the training data and determine under which conditions a model correctly captures gender information or, on the contrary, appears gender-biased.

Cited by

• Voice, Bias, and Coreference: An Interpretability Study of Gender in Speech Translation
• Social Bias in Multilingual Language Models: A Survey

Related

• Investigating grammatical abstraction in language models using few-shot learning of novel noun gender
• The Birth of Bias: A case study on the evolution of gender bias in an English language model
• Man Made Language Models? Evaluating LLMs' Perpetuation of Masculine Generics Bias
• GeNRe: A French Gender-Neutral Rewriting System Using Collective Nouns
• Identifying and Reducing Gender Bias in Word-Level Language Models