SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)

2019/03/19 by Marcos Zampieri, Zampieri, Marcos, Shervin Malmasi +10 · 22 citations
Computer Science · Psychology · Social Sciences · #Bullying, Victimization, and Aggression #Computation and Language (cs.CL) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection #Swearing, Euphemism, Multilingualism #cs.CL

paper · pdf · doi:10.48550/arxiv.1903.08983

Proceedings of the International Workshop on Semantic Evaluation (SemEval)

openalex publication_date 2019/03/19 · arxiv created 2019/04/27 · arxiv updated 2019/04/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present the results and the main findings of SemEval-2019 Task 6 on Identifying and Categorizing Offensive Language in Social Media (OffensEval). The task was based on a new dataset, the Offensive Language Identification Dataset (OLID), which contains over 14,000 English tweets. It featured three sub-tasks. In sub-task A, the goal was to discriminate between offensive and non-offensive posts. In sub-task B, the focus was on the type of offensive content in the post. Finally, in sub-task C, systems had to detect the target of the offensive posts. OffensEval attracted a large number of participants and it was one of the most popular tasks in SemEval-2019. In total, about 800 teams signed up to participate in the task, and 115 of them submitted results, which we present and analyze in this report.

Citations

• Towards NLP with Deep Learning: Convolutional Neural Networks and Recurrent Neural Networks for Offensive Language Identification in Social Media
• Enriching Word Vectors with Subword Information

Cited by

• Extending a Parliamentary Corpus with MPs' Tweets: Automatic Annotation and Evaluation Using MultiParTweet
• Patronus: Identifying and Mitigating Transferable Backdoors in Pre-trained Language Models
• Concept-Based Interpretability for Toxicity Detection
• Investigating Large Language Models' Linguistic Abilities for Text Preprocessing
• Bridging gaps in hate speech detection: meta-collections and benchmarks for low-resource Iberian languages
• Defining, Understanding, and Detecting Online Toxicity: Challenges and Machine Learning Approaches
• Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLM
• Unveiling Decision-Making in LLMs for Text Classification : Extraction of influential and interpretable concepts with Sparse Autoencoders
• Offensive Language Detection on Social Media Using XLNet
• Plug-in and Fine-tuning: Bridging the Gap between Small Language Models and Large Language Models
• Don't Patronize Me! An Annotated Dataset with Patronizing and Condescending Language towards Vulnerable Communities
• On Analyzing Antisocial Behaviors Amid COVID-19 Pandemic
• Culture Matters in Toxic Language Detection in Persian
• LLM in the Loop: Creating the ParaDeHate Dataset for Hate Speech Detoxification
• Unified Game Moderation: Soft-Prompting and LLM-Assisted Label Transfer for Resource-Efficient Toxicity Detection
• Context-Aware Content Moderation for German Newspaper Comments
• Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized Embeddings
• A Multi-Task Benchmark for Abusive Language Detection in Low-Resource Settings
• Emotions in the Loop: A Survey of Affective Computing for Emotional Support
• On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performance
• Trust The Typical
• Leveraging Affective Bidirectional Transformers for Offensive Language Detection

Related

• Duluth at SemEval-2019 Task 6: Lexical Approaches to Identify and Categorize Offensive Tweets
• SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)
• Combination of multiple Deep Learning architectures for Offensive Language Detection in Tweets
• Identifying Offensive Posts and Targeted Offense from Twitter
• Identifying and Categorizing Offensive Language in Social Media