LLM in the Loop: Creating the ParaDeHate Dataset for Hate Speech Detoxification

2025/06/02 by Shuzhou Yuan, Ercong Nie, Yuan, Shuzhou +11 · 2 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Hate Speech and Cyberbullying Detection

paper · pdf · doi:10.48550/arxiv.2506.01484

openalex publication_date 2025/06/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Detoxification, the task of rewriting harmful language into non-toxic text, has become increasingly important amid the growing prevalence of toxic content online. However, high-quality parallel datasets for detoxification, especially for hate speech, remain scarce due to the cost and sensitivity of human annotation. In this paper, we propose a novel LLM-in-the-loop pipeline leveraging GPT-4o-mini for automated detoxification. We first replicate the ParaDetox pipeline by replacing human annotators with an LLM and show that the LLM performs comparably to human annotation. Building on this, we construct ParaDeHate, a large-scale parallel dataset specifically for hatespeech detoxification. We release ParaDeHate as a benchmark of over 8K hate/non-hate text pairs and evaluate a wide range of baseline methods. Experimental results show that models such as BART, fine-tuned on ParaDeHate, achieve better performance in style accuracy, content preservation, and fluency, demonstrating the effectiveness of LLM-generated detoxification text as a scalable alternative to human annotation.

Citations

• From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge
• Style-Specific Neurons for Steering LLMs in Text Style Transfer
• Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge
• BMIKE-53: Investigating Cross-Lingual Knowledge Editing with In-Context Learning
• MultiParaDetox: Extending Text Detoxification with Parallel Data to New Languages
• Large Language Models for Data Annotation and Synthesis: A Survey
• Humans or LLMs as the Judge? A Study on Judgement Biases
• MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark
• A Ship of Theseus: Curious Cases of Paraphrasing in LLM-Generated Texts
• Making Large Language Models Better Data Creators
• LLMaAA: Making Large Language Models as Active Annotators
• Exploring Cross-Cultural Differences in English Hate Speech Annotations: From Dataset Construction to Analysis
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• Revisiting Relation Extraction in the era of Large Language Models
• LongForm: Effective Instruction Tuning with Reverse Instructions
• ChatGPT outperforms crowd workers for text-annotation tasks
• ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and\n Implicit Hate Speech Detection
• Large-Scale Hate Speech Detection with Cross-Domain Transfer
• Text Detoxification using Large Pre-trained Neural Models
• XFORMAL: A Benchmark for Multilingual Formality Style Transfer
• Civil Rephrases Of Toxic Texts With Self-Supervised Transformers
• HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection
• Deep Learning for Text Style Transfer: A Survey
• Towards A Friendly Online Community: An Unsupervised Style Transfer\n Framework for Profanity Redaction
• Language-agnostic BERT Sentence Embedding
• Parallel Data Augmentation for Formality Style Transfer
• A Probabilistic Formulation of Unsupervised Text Style Transfer
• Automatically Neutralizing Subjective Bias in Text
• BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• "Mask and Infill" : Applying Masked Language Model to Sentiment Transfer
• RoBERTa: A Robustly Optimized BERT Pretraining Approach
• A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer
• SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)
• Neural Network Acceptability Judgments
• Fighting Offensive Language on Social Media with Unsupervised Text Style Transfer
• Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer
• Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer
• Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior
• Style Transfer in Text: Exploration and Evaluation
• Style Transfer from Non-Parallel Text by Cross-Alignment
• Automated Hate Speech Detection and the Problem of Offensive Language

Cited by

• Beyond Over-Refusal: Scenario-Based Diagnostics and Post-Hoc Mitigation for Exaggerated Refusals in LLMs
• CoDAE: Adapting Large Language Models for Education via Chain-of-Thought Data Augmentation

Related

• So let's replace this phrase with insult... Lessons learned from generation of toxic texts with LLMs
• Text Detoxification: Data Efficiency, Semantic Preservation and Model Generalization
• SynthDetoxM: Modern LLMs are Few-Shot Parallel Detoxification Data Annotators
• MultiParaDetox: Extending Text Detoxification with Parallel Data to New Languages
• Detoxifying Text with MaRCo: Controllable Revision with Experts and Anti-Experts