Several Experiments on Investigating Pretraining and Knowledge-Enhanced Models for Natural Language Inference

2019/04/27 by Tianda Li, Xiaodan Zhu, Li, Tianda +11
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Speech and dialogue systems #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1904.12104

openalex publication_date 2019/04/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Natural language inference (NLI) is among the most challenging tasks in natural language understanding. Recent work on unsupervised pretraining that leverages unsupervised signals such as language-model and sentence prediction objectives has shown to be very effective on a wide range of NLP problems. It would still be desirable to further understand how it helps NLI; e.g., if it learns artifacts in data annotation or instead learn true inference knowledge. In addition, external knowledge that does not exist in the limited amount of NLI training data may be added to NLI models in two typical ways, e.g., from human-created resources or an unsupervised pretraining paradigm. We runs several experiments here to investigate whether they help NLI in the same way, and if not,how?

Related

• Unsupervised Pre-training with Structured Knowledge for Improving Natural Language Inference
• Supervising Model Attention with Human Explanations for Robust Natural Language Inference
• Unsupervised Natural Language Inference Using PHL Triplet Generation
• Lost in Inference: Rediscovering the Role of Natural Language Inference for Large Language Models
• Unsupervised Pre-training for Natural Language Generation: A Literature Review