SGHateCheck: Functional Tests for Detecting Hate Speech in Low-Resource Languages of Singapore

2024/05/03 by Ri Chi Ng, Nirmalendu Prakash, Ng, Ri Chi +7 · 4 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection

paper · pdf · doi:10.48550/arxiv.2405.01842

openalex publication_date 2024/05/03 · openalex created_date 2024/05/10 · openalex updated_date 2026/07/28

Abstract

To address the limitations of current hate speech detection models, we introduce \textsfSGHateCheck, a novel framework designed for the linguistic and cultural context of Singapore and Southeast Asia. It extends the functional testing approach of HateCheck and MHC, employing large language models for translation and paraphrasing into Singapore's main languages, and refining these with native annotators. \textsfSGHateCheck reveals critical flaws in state-of-the-art models, highlighting their inadequacy in sensitive content moderation. This work aims to foster the development of more effective hate speech detection tools for diverse linguistic environments, particularly for Singapore and Southeast Asia contexts.

Cited by

• SEA-SafeguardBench: Evaluating AI Safety in SEA Languages and Cultures
• Toxicity Red-Teaming: Benchmarking LLM Safety in Singapore's Low-Resource Languages
• LionGuard 2: Building Lightweight, Data-Efficient & Localised Multilingual Content Moderators
• Lost in Localization: Building RabakBench with Human-in-the-Loop Validation to Measure Multilingual Safety Gaps

Related

• Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models
• Hate Speech Detection in Limited Data Contexts using Synthetic Data Generation
• Hate Speech and Offensive Content Detection in Indo-Aryan Languages: A Battle of LSTM and Transformers
• Rethinking Hate Speech Detection on Social Media: Can LLMs Replace Traditional Models?
• A Survey on Automatic Online Hate Speech Detection in Low-Resource Languages