2019/07/09 by Yonatan Belinkov, Adam Poliak, Belinkov, Yonatan +7 · 2 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
paper · pdf · doi:10.48550/arxiv.1907.04380
openalex publication_date 2019/07/09 · openalex created_date 2022/07/28 · openalex updated_date 2026/07/28
Natural Language Inference (NLI) datasets often contain hypothesis-only\nbiases---artifacts that allow models to achieve non-trivial performance without\nlearning whether a premise entails a hypothesis. We propose two probabilistic\nmethods to build models that are more robust to such biases and better transfer\nacross datasets. In contrast to standard approaches to NLI, our methods predict\nthe probability of a premise given a hypothesis and NLI label, discouraging\nmodels from ignoring the premise. We evaluate our methods on synthetic and\nexisting NLI datasets by training on datasets containing biases and testing on\ndatasets containing no (or different) hypothesis-only biases. Our results\nindicate that these methods can make NLI models more robust to dataset-specific\nartifacts, transferring better than a baseline architecture in 9 out of 12 NLI\ndatasets. Additionally, we provide an extensive analysis of the interplay of\nour methods with known biases in NLI datasets, as well as the effects of\nencouraging models to ignore biases and fine-tuning on target datasets.\n