e-SNLI: Natural Language Inference with Natural Language Explanations

2018/12/04 by Oana-Maria Camburu, Tim Rocktäschel, Camburu, Oana-Maria +5 · 282 citations
Computer Science · #Artificial intelligence #Comprehension approach #Computation and Language (cs.CL) #Computer science #Domain (mathematical analysis) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Inference #Language model #Logical consequence #Natural (archaeology) #Natural Language Processing Techniques #Natural language #Natural language generation #Natural language processing #Natural language programming #Natural language understanding #Process (computing) #Sentence #Topic Modeling #Universal Networking Language #cs.CL

paper · pdf · doi:10.48550/arxiv.1812.01193

published in arXiv (Cornell University) 31, 9539-9549 (Cornell University) · NeurIPS 2018

openalex publication_date 2018/12/04 · arxiv created 2018/12/06 · arxiv updated 2018/12/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In order for machine learning to garner widespread public adoption, models must be able to provide interpretable and robust explanations for their decisions, as well as learn from human-provided explanations at train time. In this work, we extend the Stanford Natural Language Inference dataset with an additional layer of human-annotated natural language explanations of the entailment relations. We further implement models that incorporate these explanations into their training process and output them at test time. We show how our corpus of explanations, which we call e-SNLI, can be used for various goals, such as obtaining full sentence justifications of a model's decisions, improving universal sentence representations and transferring to out-of-domain NLI datasets. Our dataset thus opens up a range of research directions for using natural language explanations, both for improving models and for asserting their trust.

Cited by

• The Effectiveness of Approximate Regularized Replay for Efficient Supervised Fine-Tuning of Large Language Models
• Reason2Decide: Rationale-Driven Multi-Task Learning
• Training LLMs with LogicReward for Faithful and Rigorous Reasoning
• Explaining the Reasoning of Large Language Models Using Attribution Graphs
• Cognitive Mirrors: Exploring the Diverse Functional Roles of Attention Heads in LLM Reasoning
• Can LLMs Faithfully Explain Themselves in Low-Resource Languages? A Case Study on Emotion Detection in Persian
• Learning from Sufficient Rationales: Analysing the Relationship Between Explanation Faithfulness and Token-level Regularisation Strategies
• T-FIX: Text-Based Explanations with Features Interpretable to eXperts
• Multi-Step Knowledge Interaction Analysis via Rank-2 Subspace Disentanglement
• Agree, Disagree, Explain: Decomposing Human Label Variation in NLI through the Lens of Explanations
• SentiMaithili: A Benchmark Dataset for Sentiment and Reason Generation for the Low-Resource Maithili Language
• Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanism
• Everything is Plausible: Investigating the Impact of LLM Rationales on Human Notions of Plausibility
• VideoNorms: Benchmarking Cultural Awareness of Video Language Models
• Bridging Collaborative Filtering and Large Language Models with Dynamic Alignment, Multimodal Fusion and Evidence-grounded Explanations
• Self-Filtered Distillation with LLMs-generated Trust Indicators for Reliable Patent Classification
• o-MEGA: Optimized Methods for Explanation Generation and Analysis
• Towards Transparent AI: A Survey on Explainable Language Models
• Dual-Head Reasoning Distillation: Improving Classifier Accuracy with Train-Time-Only Reasoning
• PEPS: Quantum-Inspired Reinforcement Learning for Coherent Reasoning Traces in LLMs
• Analyzing Uncertainty of LLM-as-a-Judge: Interval Evaluations with Conformal Prediction
• Extractive Fact Decomposition for Interpretable Natural Language Inference in one Forward Pass
• Quantifying Uncertainty in Natural Language Explanations of Large Language Models for Question Answering
• Do LLMs Adhere to Label Definitions? Examining Their Receptivity to External Label Definitions
• GIER: Gap-Driven Self-Refinement for Large Language Models
• Measuring and Improving Consistency in Pretrained Language Models
• Better Language Model-Based Judging Reward Modeling through Scaling Comprehension Boundaries
• Evaluating and Characterizing Human Rationales
• Adversarial Attacks on VQA-NLE: Exposing and Alleviating Inconsistencies in Visual Question Answering Explanations
• Are Training Resources Insufficient? Predict First Then Explain!
• Can LLM-Generated Textual Explanations Enhance Model Classification Performance? An Empirical Study
• Beyond Technocratic XAI: The Who, What & How in Explanation Design
• Decoding the Multimodal Maze: A Systematic Review on the Adoption of Explainability in Multimodal Attention-based Models
• Alleviating Attention Hacking in Discriminative Reward Modeling through Interaction Distillation
• ISO-Bench: Benchmarking Multimodal Causal Reasoning in Visual-Language Models through Procedural Plans
• Metaphor and Large Language Models: When Surface Features Matter More than Deep Understanding
• Order Constraints in Optimal Transport
• Filling the Gap: Is Commonsense Knowledge Generation useful for Natural Language Inference?
• What Gets Echoed? Understanding the "Pointers" in Explanations of Persuasive Arguments
• Two-Stage Reasoning-Infused Learning: Improving Classification with LLM-Generated Reasoning
• ARGUS: Hallucination and Omission Evaluation in Video-LLMs
• Evaluating Causal Explanation in Medical Reports with LLM-Based and Human-Aligned Metrics
• Investigating the Effect of Natural Language Explanations on Out-of-Distribution Generalization in Few-shot NLI
• Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?
• Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety
• A Survey on Explainability in Machine Reading Comprehension
• Rationales Are Not Silver Bullets: Measuring the Impact of Rationales on Model Performance and Reliability
• CrossICL: Cross-Task In-Context Learning via Unsupervised Demonstration Transfer
• Faithful and Robust LLM-Driven Theorem Proving for NLI Explanations
• Self-Critique and Refinement for Faithful Natural Language Explanations
• LiTEx: A Linguistic Taxonomy of Explanations for Understanding Within-Label Variation in Natural Language Inference
• Leveraging Large Language Models for Bengali Math Word Problem Solving with Chain of Thought Reasoning
• Counterfactual Simulatability of LLM Explanations for Generation Tasks
• Improving the OOD Performance of Closed-Source LLMs on NLI Through Strategic Data Selection
• On Sample Based Explanation Methods for NLP:Efficiency, Faithfulness, and Semantic Evaluation
• A Necessary Step toward Faithfulness: Measuring and Improving Consistency in Free-Text Explanations
• The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT Distillation
• You Can Do Better! If You Elaborate the Reason When Making Prediction
• Trustworthy AI in Digital Health: A Comprehensive Review of Robustness and Explainability
• Thinking Aloud: Dynamic Context Generation Improves Zero-Shot Reasoning Performance of GPT-2
• Abductive Commonsense Reasoning
• Latent Veracity Inference for Identifying Errors in Stepwise Reasoning
• Explainable Machine Learning with Prior Knowledge: An Overview
• Towards Explainable NLP: A Generative Explanation Framework for Text Classification
• Towards Explainable Fact Checking
• Unpacking Robustness in Inflectional Languages: Adversarial Evaluation and Mechanistic Insights
• EvalxNLP: A Framework for Benchmarking Post-Hoc Explainability Methods on NLP Models
• From Evidence to Belief: A Bayesian Epistemology Approach to Language Models
• Learning to Draw ASCII Improves Spatial Reasoning in Language Models
• Pushing the boundary on Natural Language Inference
• Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers
• Honey, I Shrunk the Language Model: Impact of Knowledge Distillation Methods on Performance and Explainability
• ChartQA-X: Generating Explanations for Visual Chart Reasoning
• PEIRCE: Unifying Material and Formal Reasoning via LLM-Driven Neuro-Symbolic Refinement
• A Survey on Efficient Vision-Language Models
• Cross-Document Cross-Lingual NLI via RST-Enhanced Graph Fusion and Interpretability Prediction
• Model-Agnostic Policy Explanations with Large Language Models

Related

• Logical Reasoning with Span-Level Predictions for Interpretable and Robust NLI Models
• Generating Token-Level Explanations for Natural Language Inference
• Does External Knowledge Help Explainable Natural Language Inference? Automatic Evaluation vs. Human Ratings
• Do Natural Language Explanations Represent Valid Logical Arguments? Verifying Entailment in Explainable NLI Gold Standards
• Reasoning with Natural Language Explanations