A Convolutional Neural Network for Modelling Sentences

2014/04/08 by Nal Kalchbrenner, Edward Grefenstette, Kalchbrenner, Nal +3 · 1 voice · 17 citations
Computer Science · #Natural Language Processing Techniques #Sentiment Analysis and Opinion Mining #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.1404.2188

openalex publication_date 2014/04/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The ability to accurately represent sentences is central to language understanding. We describe a convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) that we adopt for the semantic modelling of sentences. The network uses Dynamic k-Max Pooling, a global pooling operation over linear sequences. The network handles input sentences of varying length and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations. The network does not rely on a parse tree and is easily applicable to any language. We test the DCNN in four experiments: small scale binary and multi-class sentiment prediction, six-way question classification and Twitter sentiment prediction by distant supervision. The network achieves excellent performance in the first three tasks and a greater than 25% error reduction in the last task with respect to the strongest baseline.

Citations

• Learning to Map Sentences to Logical Form: Structured Classification\n with Probabilistic Categorial Grammars
• Category-Theoretic Quantitative Compositional Distributional Models of Natural Language Semantics
• Improving neural networks by preventing co-adaptation of feature detectors
• Gradient-based learning applied to document recognition

Cited by

• A Novel Graph-Sequence Learning Model for Inductive Text Classification
• GFT: Graph Feature Tuning for Efficient Point Cloud Analysis
• DWFF-Net : A Multi-Scale Farmland System Habitat Identification Method with Adaptive Dynamic Weight
• Dictionary-Based Deblurring for Unpaired Data
• Towards Speeding up Program Repair with Non-Autoregressive Model
• Causal Multi-fidelity Surrogate Forward and Inverse Models for ICF Implosions
• Knowledge Graph Representation with Jointly Structural and Textual Encoding
• Jointly Generating and Attributing Answers using Logits of Document-Identifier Tokens
• On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Survey
• CIS at TAC Cold Start 2015: Neural Networks and Coreference Resolution for Slot Filling
• Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms
• A Survey on Text Classification: From Shallow to Deep Learning
• Pre-Trained Models: Past, Present and Future
• Semantic-Unit-Based Dilated Convolution for Multi-Label Text Classification
• Integrating Physiological Time Series and Clinical Notes with Deep\n Learning for Improved ICU Mortality Prediction
• Modelling Interaction of Sentence Pair with coupled-LSTMs
• Convolutional neural network [wikipedia]

Discussions

• A Convolutional Neural Network for Modelling Sentences (2014) [pdf] [hn, 62 points, 22 comments]

Related

• Multichannel Variable-Size Convolution for Sentence Classification
• Dependency-based Convolutional Neural Networks for Sentence Embedding
• Convolutional Neural Networks for Sentence Classification
• Discriminative Neural Sentence Modeling by Tree-Based Convolution
• Learning Generic Sentence Representations Using Convolutional Neural Networks