Distributions In, Distributions Out: The Case for Soft-Label Training

2025/11/18 by Agamdeep Singh, Singh, Agamdeep, Ashish Tiwari +5
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2511.14117

arxiv created 2026/07/30 · arxiv updated 2026/07/31

Abstract

Supervised classifiers output a distribution over classes but are typically trained against a single label obtained by collapsing multiple annotators into a majority vote. On tasks where annotator disagreement reflects genuine ambiguity -- natural language inference, politeness, visually ambiguous categorization -- this collapse discards information and forces models to express uniform confidence on inputs where humans systematically disagree. We compare soft-label training, which uses the full annotation distribution as the target, against hard-label training across three datasets spanning vision and NLP (ChaosNLI, POPQUORN, CIFAR-10H). Soft-label training matches or exceeds hard-label accuracy on every dataset, reduces KL divergence to the annotator distribution by 32% on average (p < 10^-4), and produces predictions whose per-sample entropy correlates 61% more strongly with annotator entropy -- models trained on distributions are uncertain precisely where humans are. We argue these benefits follow from a basic observation: when annotators legitimately disagree, the annotation distribution is the correct learning target, not a noisy estimate of it.

Related

• Learning with Confidence: Training Better Classifiers from Soft Labels
• Perceptual Quality-based Model Training under Annotator Label Uncertainty
• Soft Label Memorization-Generalization for Natural Language Inference
• Improving Label Quality by Jointly Modeling Items and Annotators
• Characterizing Sources of Uncertainty to Proxy Calibration and\n Disambiguate Annotator and Data Bias