Cross-lingual Zero- and Few-shot Hate Speech Detection Utilising Frozen Transformer Language Models and AXEL

2020/04/13 by Lukas Stappen, Stappen, Lukas, Fabian Brunn +4 · 1 citation
Computer Science · Mathematics · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection #Machine Learning (cs.LG) #Machine Learning (stat.ML) #cs.CL #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2004.13850

arxiv created 2020/04/13 · openalex publication_date 2020/04/13 · arxiv updated 2020/04/30 · openalex created_date 2020/05/13 · openalex updated_date 2026/07/28

Abstract

Detecting hate speech, especially in low-resource languages, is a non-trivial challenge. To tackle this, we developed a tailored architecture based on frozen, pre-trained Transformers to examine cross-lingual zero-shot and few-shot learning, in addition to uni-lingual learning, on the HatEval challenge data set. With our novel attention-based classification block AXEL, we demonstrate highly competitive results on the English and Spanish subsets. We also re-sample the English subset, enabling additional, meaningful comparisons in the future.

Citations

• Long Short-Term Memory
• Word Translation Without Parallel Data
• Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond
• Cross-lingual Language Model Pretraining
• Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT
• Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

Cited by

• Bayesian Methods for Semi-supervised Text Annotation

Related

• Deep Learning Models for Multilingual Hate Speech Detection
• Multilingual Auxiliary Tasks Training: Bridging the Gap between Languages for Zero-Shot Transfer of Hate Speech Detection Models
• Addressing the Challenges of Cross-Lingual Hate Speech Detection
• Can Prompting LLMs Unlock Hate Speech Detection across Languages? A Zero-shot and Few-shot Study
• Cross-lingual Hate Speech Detection using Transformer Models