Sense Embeddings are also Biased--Evaluating Social Biases in Static and\n Contextualised Sense Embeddings

2022/03/14 by Yi Zhou, Masahiro Kaneko, Zhou, Yi +3 · 2 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Sentiment Analysis and Opinion Mining #Speech and dialogue systems #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2203.07523

openalex publication_date 2022/03/14 · openalex created_date 2022/05/05 · openalex updated_date 2026/07/28

Abstract

Sense embedding learning methods learn different embeddings for the different\nsenses of an ambiguous word. One sense of an ambiguous word might be socially\nbiased while its other senses remain unbiased. In comparison to the numerous\nprior work evaluating the social biases in pretrained word embeddings, the\nbiases in sense embeddings have been relatively understudied. We create a\nbenchmark dataset for evaluating the social biases in sense embeddings and\npropose novel sense-specific bias evaluation measures. We conduct an extensive\nevaluation of multiple static and contextualised sense embeddings for various\ntypes of social biases using the proposed measures. Our experimental results\nshow that even in cases where no biases are found at word-level, there still\nexist worrying levels of social biases at sense-level, which are often ignored\nby the word-level bias evaluation measures.\n

Cited by

• Gamma Mixture Modeling for Cosine Similarity in Small Language Models
• Exploring Gender Bias Beyond Occupational Titles

Related

• Measuring Social Biases in Grounded Vision and Language Embeddings
• Learning Sense-Specific Static Embeddings using Contextualised Word Embeddings as a Proxy
• On Measuring Social Biases in Sentence Encoders
• Lost in Context? On the Sense-wise Variance of Contextualized Word Embeddings
• When do Word Embeddings Accurately Reflect Surveys on our Beliefs About\n People?