Investigating the Effect of Natural Language Explanations on Out-of-Distribution Generalization in Few-shot NLI

2021/10/12 by Yangqiaoyu Zhou, Chenhao Tan, Zhou, Yangqiaoyu +1 · 2 citations
Computer Science · Mathematics · #Artificial Intelligence (cs.AI) #Artificial intelligence #Computation and Language (cs.CL) #Computer science #Construct (python library) #FOS: Computer and information sciences #Generalization #Hallucinating #Leverage (statistics) #Machine learning #Mathematics #Multimodal Machine Learning Applications #Natural (archaeology) #Natural Language Processing Techniques #Natural language #Natural language processing #Topic Modeling #cs.AI #cs.CL

paper · pdf · doi:10.48550/arxiv.2110.06223

published in arXiv (Cornell University) (Cornell University) · 8 pages, 4 figures, EMNLPWorkshop on Insights from Negative Results in NLP 2021, data is available at https://github.com/ChicagoHAI/hans-explanations

arxiv created 2021/10/12 · openalex publication_date 2021/10/12 · arxiv updated 2021/10/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Although neural models have shown strong performance in datasets such as SNLI, they lack the ability to generalize out-of-distribution (OOD). In this work, we formulate a few-shot learning setup and examine the effects of natural language explanations on OOD generalization. We leverage the templates in the HANS dataset and construct templated natural language explanations for each template. Although generated explanations show competitive BLEU scores against groundtruth explanations, they fail to improve prediction performance. We further show that generated explanations often hallucinate information and miss key elements that indicate the label.

Citations

• e-SNLI: Natural Language Inference with Natural Language Explanations
• WT5?! Training Text-to-Text Models to Explain their Predictions
• Adversarial Filters of Dataset Biases

Cited by

• Do Generalisation Results Generalise?
• Bayesian Cross-Modal Alignment Learning for Few-Shot Out-of-Distribution Generalization

Related

• FLamE: Few-shot Learning from Natural Language Explanations
• Few-Shot Out-of-Domain Transfer Learning of Natural Language Explanations in a Label-Abundant Setup
• Supervising Model Attention with Human Explanations for Robust Natural Language Inference
• Self-Rationalization in the Wild: A Large Scale Out-of-Distribution Evaluation on NLI-related tasks
• Does External Knowledge Help Explainable Natural Language Inference? Automatic Evaluation vs. Human Ratings