Demographic Dialectal Variation in Social Media: A Case Study of\n African-American English

2016/08/31 by Su Lin Blodgett, Blodgett, Su Lin, Lisa Green +3 · 14 citations
Arts and Humanities · Social Sciences · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Linguistic Variation and Morphology #Linguistics, Language Diversity, and Identity #Multilingual Education and Policy

paper · pdf · doi:10.48550/arxiv.1608.08868

openalex publication_date 2016/08/31 · openalex created_date 2022/10/03 · openalex updated_date 2026/07/28

Abstract

Though dialectal language is increasingly abundant on social media, few\nresources exist for developing NLP tools to handle such language. We conduct a\ncase study of dialectal language in online conversational text by investigating\nAfrican-American English (AAE) on Twitter. We propose a distantly supervised\nmodel to identify AAE-like language from demographics associated with\ngeo-located messages, and we verify that this language follows well-known AAE\nlinguistic phenomena. In addition, we analyze the quality of existing language\nidentification and dependency parsing tools on AAE-like text, demonstrating\nthat they perform poorly on such text compared to text associated with white\nspeakers. We also provide an ensemble classifier for language identification\nwhich eliminates this disparity and release a new corpus of tweets containing\nAAE-like language.\n

Cited by

• BAID: A Benchmark for Bias Assessment of AI Detectors
• BERnaT: Basque Encoders for Representing Natural Textual Diversity
• Technical Report on Text Dataset Distillation
• How AI Fails: An Interactive Pedagogical Tool for Demonstrating Dialectal Bias in Automated Toxicity Models
• Are Stereotypes Leading LLMs' Zero-Shot Stance Detection ?
• Identity-Aware Large Language Models require Cultural Reasoning
• Artificial Impressions: Evaluating Large Language Model Behavior Through the Lens of Trait Impressions
• Less Is More? Examining Fairness in Pruned Large Language Models for Summarising Opinions
• I Think, Therefore I Am Under-Qualified? A Benchmark for Evaluating Linguistic Shibboleth Detection in LLM Hiring Evaluations
• Benchmarking Sociolinguistic Diversity in Swahili NLP: A Taxonomy-Guided Approach
• Language Models Change Facts Based on the Way You Talk
• Nonlinear Concept Erasure: a Density Matching Approach
• When Large Language Models Meet Law: Dual-Lens Taxonomy, Technical Advances, and Ethical Governance
• Steps Adaptive Decay DPSGD: Enhancing Performance on Imbalanced Datasets with Differential Privacy with HAM10000

Related

• Towards a Deep Multi-layered Dialectal Language Analysis: A Case Study of African-American English
• Towards Better Inclusivity: A Diverse Tweet Corpus of English Varieties
• Racial Disparity in Natural Language Processing: A Case Study of Social Media African-American English
• Sociolinguistic Analysis with Missing Metadata? Leveraging Linguistic and Semiotic Resources Through Deep Learning to Investigate English Variation and Change on Twitter
• Characterizing Interconnections and Linguistic Patterns in Twitter