Investigating the Effect of Natural Language Explanations on Out-of-Distribution Generalization in Few-shot NLI

2021/10/12 by Yangqiaoyu Zhou, Chenhao Tan, Zhou, Yangqiaoyu +1 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2110.06223

openalex publication_date 2021/10/12 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Although neural models have shown strong performance in datasets such as SNLI, they lack the ability to generalize out-of-distribution (OOD). In this work, we formulate a few-shot learning setup and examine the effects of natural language explanations on OOD generalization. We leverage the templates in the HANS dataset and construct templated natural language explanations for each template. Although generated explanations show competitive BLEU scores against groundtruth explanations, they fail to improve prediction performance. We further show that generated explanations often hallucinate information and miss key elements that indicate the label.

Citations

• e-SNLI: Natural Language Inference with Natural Language Explanations
• WT5?! Training Text-to-Text Models to Explain their Predictions
• Adversarial Filters of Dataset Biases

Cited by

• Do Generalisation Results Generalise?

Related

• FLamE: Few-shot Learning from Natural Language Explanations
• Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations in a Label-Abundant Setup
• Supervising Model Attention with Human Explanations for Robust Natural Language Inference
• Self-Rationalization in the Wild: A Large Scale Out-of-Distribution Evaluation on NLI-related tasks
• Does External Knowledge Help Explainable Natural Language Inference?\n Automatic Evaluation vs. Human Ratings