2019/09/06 by Tongfei Chen, Zhengping Jiang, Chen, Tongfei +7 · 3 citations
Computer Science · #Artificial intelligence #Categorical variable #Computation and Language (cs.CL) #Computer science #FOS: Computer and information sciences #I.2.7 #Inference #Linguistics #Machine learning #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Natural language #Natural language processing #Premise #Probabilistic logic #Topic Modeling #cs.CL
paper · pdf · doi:10.48550/arxiv.1909.03042
published in arXiv (Cornell University) (Cornell University) · Accepted to ACL 2020
openalex publication_date 2019/09/06 · arxiv created 2020/05/05 · arxiv updated 2020/05/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We introduce Uncertain Natural Language Inference (UNLI), a refinement of Natural Language Inference (NLI) that shifts away from categorical labels, targeting instead the direct prediction of subjective probability assessments. We demonstrate the feasibility of collecting annotations for UNLI by relabeling a portion of the SNLI dataset under a probabilistic scale, where items even with the same categorical label differ in how likely people judge them to be true given a premise. We describe a direct scalar regression modeling approach, and find that existing categorically labeled NLI data can be used in pre-training. Our best models approach human performance, demonstrating models may be capable of more subtle inferences than the categorical bin assignment employed in current NLI tasks.