Shortcut-Stacked Sentence Encoders for Multi-Domain Inference

2017/08/07 by Yixin Nie, Mohit Bansal, Nie, Yixin +1 · 4 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1708.02312

openalex publication_date 2017/08/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present a simple sequential sentence encoder for multi-domain natural language inference. Our encoder is based on stacked bidirectional LSTM-RNNs with shortcut connections and fine-tuning of word embeddings. The overall supervised model uses the above encoder to encode two input sentences into two vectors, and then uses a classifier over the vector combination to label the relationship between these two sentences as that of entailment, contradiction, or neural. Our Shortcut-Stacked sentence encoders achieve strong improvements over existing encoders on matched and mismatched multi-domain natural language inference (top non-ensemble single-model result in the EMNLP RepEval 2017 Shared Task (Nangia et al., 2017)). Moreover, they achieve the new state-of-the-art encoding result on the original SNLI dataset (Bowman et al., 2015).

Citations

• A large annotated corpus for learning natural language inference

Cited by

• Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information
• Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling
• Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms
• Dynamic Compositionality in Recursive Neural Networks with Structure-aware Tag Representations

Related

• Recurrent Neural Network-Based Sentence Encoder with Gated Attention for Natural Language Inference
• The RepEval 2017 Shared Task: Multi-Genre Natural Language Inference with Sentence Representations
• Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation
• First Train to Generate, then Generate to Train: UnitedSynT5 for Few-Shot NLI
• Natural Language Inference from Multiple Premises