Trade-Offs Between Fairness and Privacy in Language Modeling

2023/05/24 by Cleo Matzken, Steffen Eger, Matzken, Cleo +3
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection

paper · pdf · doi:10.48550/arxiv.2305.14936

openalex publication_date 2023/05/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Protecting privacy in contemporary NLP models is gaining in importance. So does the need to mitigate social biases of such models. But can we have both at the same time? Existing research suggests that privacy preservation comes at the price of worsening biases in classification tasks. In this paper, we explore the extent to which this tradeoff really holds when we incorporate both privacy preservation and de-biasing techniques into training text generation models. How does improving the model along one dimension affect the other dimension as well as the utility of the model? We conduct an extensive set of experiments that include bias detection, privacy attacks, language modeling, and performance on downstream tasks.

Related

• Does Differential Privacy Impact Bias in Pretrained NLP Models?
• De-amplifying Bias from Differential Privacy in Language Model Fine-tuning
• Characterizing Stereotypical Bias from Privacy-preserving Pre-Training
• Fair NLP Models with Differentially Private Text Encoders
• You Are What You Write: Preserving Privacy in the Era of Large Language Models