Gender Biases Unexpectedly Fluctuate in the Pre-training Stage of Masked Language Models

2022/11/26 by Kenan Tang, Tang, Kenan, Hanchun Jiang +1 · 1 citation
Social Sciences · #Computation and Language (cs.CL) #Computational and Text Analysis Methods #FOS: Computer and information sciences #Gender Studies in Language

paper · pdf · doi:10.48550/arxiv.2211.14639

openalex publication_date 2022/11/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Masked language models pick up gender biases during pre-training. Such biases are usually attributed to a certain model architecture and its pre-training corpora, with the implicit assumption that other variations in the pre-training process, such as the choices of the random seed or the stopping point, have no effect on the biases measured. However, we show that severe fluctuations exist at the fundamental level of individual templates, invalidating the assumption. Further against the intuition of how humans acquire biases, these fluctuations are not correlated with the certainty of the predicted pronouns or the profession frequencies in pre-training corpora. We release our code and data to benefit future research.

Cited by

• Language Model Behavior: A Comprehensive Survey

Related

• Gender Encoding Patterns in Pretrained Language Model Representations
• The Birth of Bias: A case study on the evolution of gender bias in an English language model
• UnMASKed: Quantifying Gender Biases in Masked Language Models through Linguistically Informed Job Market Prompts
• Evaluating Gender Bias Transfer between Pre-trained and Prompt-Adapted Language Models
• Identifying and Reducing Gender Bias in Word-Level Language Models