Unmasking and Improving Data Credibility: A Study with Datasets for Training Harmless Language Models

2023/11/19 by Zhaowei Zhu, Zhu, Zhaowei, Jialu Wang +5 · 7 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computers and Society (cs.CY) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2311.11202

openalex publication_date 2023/11/19 · openalex created_date 2023/11/22 · openalex updated_date 2026/07/28

Abstract

Language models have shown promise in various tasks but can be affected by undesired data during training, fine-tuning, or alignment. For example, if some unsafe conversations are wrongly annotated as safe ones, the model fine-tuned on these samples may be harmful. Therefore, the correctness of annotations, i.e., the credibility of the dataset, is important. This study focuses on the credibility of real-world datasets, including the popular benchmarks Jigsaw Civil Comments, Anthropic Harmless & Red Team, PKU BeaverTails & SafeRLHF, that can be used for training a harmless language model. Given the cost and difficulty of cleaning these datasets by humans, we introduce a systematic framework for evaluating the credibility of datasets, identifying label errors, and evaluating the influence of noisy labels in the curated language data, specifically focusing on unsafe comments and conversation classification. With the framework, we find and fix an average of 6.16% label errors in 11 datasets constructed from the above benchmarks. The data credibility and downstream learning performance can be remarkably improved by directly fixing label errors, indicating the significance of cleaning existing real-world datasets. We provide an open-source tool, Docta, for data cleaning at https://github.com/Docta-ai/docta.

Cited by

• Robust Preference Alignment via Directional Neighborhood Consensus
• Rethinking Toxicity Evaluation in Large Language Models: A Multi-Label Perspective
• Better Reasoning with Less Data: Enhancing VLMs Through Unified Modality Scoring
• EXP-Bench: Can AI Conduct AI Research Experiments?
• When VLMs Meet Image Classification: Test Sets Renovation via Missing Label Identification
• Are Vision-Language Models Safe in the Wild? A Meme-Based Benchmark Study
• Exploring Criteria of Loss Reweighting to Enhance LLM Unlearning

Related

• Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models
• Detecting and Filtering Unsafe Training Data via Data Attribution with Denoised Representation
• Beyond the Safety Bundle: Auditing the Helpful and Harmless Dataset
• Learning and Forgetting Unsafe Examples in Large Language Models
• MBIAS: Mitigating Bias in Large Language Models While Retaining Context