Can Prompting LLMs Unlock Hate Speech Detection across Languages? A Zero-shot and Few-shot Study

2025/05/09 by Faeze Ghorbanpour, Daryna Dementieva, Ghorbanpour, Faeze +3 · 3 citations
Computer Science · #Computation and Language (cs.CL) #Computers and Society (cs.CY) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection #Multimedia (cs.MM) #Sentiment Analysis and Opinion Mining #Spam and Phishing Detection

paper · pdf · doi:10.48550/arxiv.2505.06149

openalex publication_date 2025/05/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Despite growing interest in automated hate speech detection, most existing approaches overlook the linguistic diversity of online content. Multilingual instruction-tuned large language models such as LLaMA, Aya, Qwen, and BloomZ offer promising capabilities across languages, but their effectiveness in identifying hate speech through zero-shot and few-shot prompting remains underexplored. This work evaluates LLM prompting-based detection across eight non-English languages, utilizing several prompting techniques and comparing them to fine-tuned encoder models. We show that while zero-shot and few-shot prompting lag behind fine-tuned encoder models on most of the real-world evaluation sets, they achieve better generalization on functional tests for hate speech detection. Our study also reveals that prompt design plays a critical role, with each language often requiring customized prompting techniques to maximize performance.

Citations

• Evaluation of Hate Speech Detection Using Large Language Models and Geographical Contextualization
• Guardians of Discourse: Evaluating LLMs on Multilingual Offensive Language Detection
• Better to Ask in English: Evaluation of Large Language Models on English, Low-resource and Cross-Lingual Settings
• Hate Personified: Investigating the role of LLMs in content moderation
• Teuken-7B-Base & Teuken-7B-Instruct: Towards European LLMs
• Improving Adversarial Data Collection by Supporting Annotators: Lessons from GAHD, a German Hate Speech Dataset
• Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model
• Multilingual Instruction Tuning With Just a Pinch of Multilinguality
• When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks
• Probing LLMs for hate speech detection: strengths and vulnerabilities
• Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models
• Large-Scale Hate Speech Detection with Cross-Domain Transfer
• Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
• XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond
• DeBERTa: Decoding-enhanced BERT with Disentangled Attention
• Unsupervised Cross-lingual Representation Learning at Scale
• Multilingual and Multi-Aspect Hate Speech Analysis

Cited by

• LLM Harms: A Taxonomy and Discussion
• On the Sensitivity of Instruction-tuned LLMs to Harmful Sentences in Long Inputs
• MMBERT: Scaled Mixture-of-Experts Multimodal BERT for Robust Chinese Hate Speech Detection under Cloaking Perturbations

Related

• LLMs and Finetuning: Benchmarking cross-domain performance for hate speech detection
• An Investigation of Large Language Models for Real-World Hate Speech Detection
• Data-Efficient Strategies for Expanding Hate Speech Detection into Under-Resourced Languages
• Evaluating the Effectiveness of Natural Language Inference for Hate Speech Detection in Languages with Limited Labeled Data
• Deep Learning Models for Multilingual Hate Speech Detection