Counterfactual Probing for the Influence of Affect and Specificity on Intergroup Bias

2023/05/25 by Venkata S Govindarajan, Kyle Mahowald, Govindarajan, Venkata S +5
Computer Science · #Computation and Language (cs.CL) #Computers and Society (cs.CY) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection #Text Readability and Simplification

paper · pdf · doi:10.48550/arxiv.2305.16409

openalex publication_date 2023/05/25 · openalex created_date 2023/05/30 · openalex updated_date 2026/07/28

Abstract

While existing work on studying bias in NLP focues on negative or pejorative language use, Govindarajan et al. (2023) offer a revised framing of bias in terms of intergroup social context, and its effects on language behavior. In this paper, we investigate if two pragmatic features (specificity and affect) systematically vary in different intergroup contexts -- thus connecting this new framing of bias to language output. Preliminary analysis finds modest correlations between specificity and affect of tweets with supervised intergroup relationship (IGR) labels. Counterfactual probing further reveals that while neural models finetuned for predicting IGR labels reliably use affect in classification, the model's usage of specificity is inconclusive. Code and data can be found at: https://github.com/venkatasg/intergroup-probing

Related

• How people talk about each other: Modeling Generalized Intergroup Bias and Emotion
• Toward Understanding Bias Correlations for Mitigation in NLP
• Social Bias Frames: Reasoning about Social and Power Implications of\n Language
• Generative Language Models Exhibit Social Identity Biases
• Subtle Biases Need Subtler Measures: Dual Metrics for Evaluating Representative and Affinity Bias in Large Language Models