Analyzing Gender Representation in Multilingual Models

2022/04/20 by Hila Gonen, Gonen, Hila, Shauli Ravfogel +3 · 2 citations
Computer Science · Social Sciences · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Gender Studies in Language #Natural Language Processing Techniques #Text Readability and Simplification

paper · pdf · doi:10.48550/arxiv.2204.09168

openalex publication_date 2022/04/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Multilingual language models were shown to allow for nontrivial transfer across scripts and languages. In this work, we study the structure of the internal representations that enable this transfer. We focus on the representation of gender distinctions as a practical case study, and examine the extent to which the gender concept is encoded in shared subspaces across different languages. Our analysis shows that gender representations consist of several prominent components that are shared across languages, alongside language-specific components. The existence of language-independent and language-specific components provides an explanation for an intriguing empirical observation we make: while gender classification transfers well across languages, interventions for gender removal, trained on a single language, do not transfer easily to others.

Cited by

• Undesirable Biases in NLP: Addressing Challenges of Measurement
• Benchmarking Gender and Political Bias in Large Language Models

Related

• Gender Lost In Translation: How Bridging The Gap Between Languages Affects Gender Bias in Zero-Shot Multilingual Translation
• Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer
• What an Elegant Bridge: Multilingual LLMs are Biased Similarly in Different Languages
• Investigating grammatical abstraction in language models using few-shot learning of novel noun gender
• Grammatical gender processing in bilinguals: An analytic review