LionGuard: Building a Contextualized Moderation Classifier to Tackle Localized Unsafe Content

2024/06/24 by Jessica Foo, Foo, Jessica, Khoo, Shaun · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection

paper · pdf · doi:10.48550/arxiv.2407.10995

openalex publication_date 2024/06/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

As large language models (LLMs) become increasingly prevalent in a wide variety of applications, concerns about the safety of their outputs have become more significant. Most efforts at safety-tuning or moderation today take on a predominantly Western-centric view of safety, especially for toxic, hateful, or violent speech. In this paper, we describe LionGuard, a Singapore-contextualized moderation classifier that can serve as guardrails against unsafe LLM outputs. When assessed on Singlish data, LionGuard outperforms existing widely-used moderation APIs, which are not finetuned for the Singapore context, by 14% (binary) and up to 51% (multi-label). Our work highlights the benefits of localization for moderation classifiers and presents a practical and scalable approach for low-resource languages.

Cited by

• Toxicity-Aware Few-Shot Prompting for Low-Resource Singlish Translation

Related

• LionGuard 2: Building Lightweight, Data-Efficient & Localised Multilingual Content Moderators
• Lost in Localization: Building RabakBench with Human-in-the-Loop Validation to Measure Multilingual Safety Gaps
• Toxicity Red-Teaming: Benchmarking LLM Safety in Singapore's Low-Resource Languages
• PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages
• MrGuard: A Multilingual Reasoning Guardrail for Universal LLM Safety