A Decomposable Attention Model for Natural Language Inference

2016/06/06 by Ankur P. Parikh, Parikh, Ankur P., Oscar Täckström +5 · 1 voice · 15 citations
Computer Science · #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.1606.01933

openalex publication_date 2016/06/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose a simple neural architecture for natural language inference. Our approach uses attention to decompose the problem into subproblems that can be solved separately, thus making it trivially parallelizable. On the Stanford Natural Language Inference (SNLI) dataset, we obtain state-of-the-art results with almost an order of magnitude fewer parameters than previous work and without relying on any word-order information. Adding intra-sentence attention that takes a minimum amount of order into account yields further improvements.

Citations

• A large annotated corpus for learning natural language inference

Cited by

• Fast weight programming and linear transformers: from machine learning to neurobiology
• Dynamical Properties of Tokens in Self-Attention and Effects of Positional Encoding
• Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQA
• Dropping Networks for Transfer Learning
• Deep Lookup Network
• Adversarial Attacks Against Automated Fact-Checking: A Survey
• Explainable Semantic Text Relations: A Question-Answering Framework for Comparing Document Content
• Frame-Semantic Parsing with Softmax-Margin Segmental RNNs and a Syntactic Scaffold
• Dynamic Multi-Level Multi-Task Learning for Sentence Simplification
• Adversarial Training for Community Question Answer Selection Based on Multi-scale Matching
• Sanity Check: A Strong Alignment and Information Retrieval Baseline for Question Answering
• Endowing Deep 3D Models with Rotation Invariance Based on Principal Component Analysis
• Recognising Agreement and Disagreement between Stances with Reason Comparing Networks
• Enhance word representation for out-of-vocabulary on Ubuntu dialogue corpus
• Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks

Discussions

• It wasn't even the transformer paper that was first to show attention was all you need. Everyone forgets how aggressively folks were working on faster alternatives to RNNs ~2016, and another paper fro [bsky, 22 points, 3 comments]

Related

• Syntax-based Attention Model for Natural Language Inference
• A Neural Architecture Mimicking Humans End-to-End for Natural Language Inference
• Attention Boosted Sequential Inference Model
• Recurrent Neural Network-Based Sentence Encoder with Gated Attention for Natural Language Inference
• Knowledge Enhanced Attention for Robust Natural Language Inference