Text Understanding from Scratch

2015/02/05 by Xiang Zhang, Yann LeCun, Zhang, Xiang +1 · 4 voices · 421 citations
Computer Science · #Advanced Text Analysis Techniques #Artificial intelligence #Categorization #Character (mathematics) #Computer science #Natural language processing #Ontology #Scratch #Sentiment Analysis and Opinion Mining #Sentiment analysis #Topic Modeling #cs.CL #cs.LG

paper · pdf · doi:10.48550/arxiv.1502.01710

published in arXiv (Cornell University) (Cornell University) · This technical report is superseded by a paper entitled "Character-level Convolutional Networks for Text Classification", arXiv:1509.01626. It has considerably more experimental results and a rewritten introduction

openalex publication_date 2015/02/05 · arxiv created 2016/04/04 · arxiv updated 2016/04/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This article demontrates that we can apply deep learning to text understanding from character-level inputs all the way up to abstract text concepts, using temporal convolutional networks (ConvNets). We apply ConvNets to various large-scale datasets, including ontology classification, sentiment analysis, and text categorization. We show that temporal ConvNets can achieve astonishing performance without the knowledge of words, phrases, sentences and any other syntactic or semantic structures with regards to a human language. Evidence shows that our models can work for both English and Chinese.

Citations

• Learning to Execute
• Learning Semantic Representations for the Phrase Translation Model
• Improving neural networks by preventing co-adaptation of feature detectors
• Long Short-Term Memory
• Gradient-based learning applied to document recognition
• Exploiting Similarities among Languages for Machine Translation
• Distributed Representations of Sentences and Documents
• Distributed Representations of Words and Phrases and their Compositionality
• ImageNet classification with deep convolutional neural networks
• Rich feature hierarchies for accurate object detection and semantic segmentation
• Character-level Convolutional Networks for Text Classification

Cited by

• VizGen: Data Exploration and Visualization from Natural Language via a Multi-Agent AI Architecture
• Adversarial Text Generation with Dynamic Contextual Perturbation
• Character-level Convolutional Networks for Text Classification
• The SP Theory of Intelligence: Distinctive Features and Advantages
• Sequence to sequence learning for unconstrained scene text recognition
• An Attentive Sequence Model for Adverse Drug Event Extraction from Biomedical Text
• Combination of Domain Knowledge and Deep Learning for Sentiment Analysis

Discussions

• Text Understanding from Scratch Using Temporal Convolutional Networks [hn, 61 points, 9 comments]
• Text Understanding from Scratch [pdf] [hn, 53 points, 5 comments]
• Text Understanding from Scratch (2015) [hn, 10 points, 0 comments]
• Text understanding from scratch [hn, 2 points, 0 comments]

Related

• Character-level Convolutional Networks for Text Classification
• Do Convolutional Networks need to be Deep for Text Classification ?
• Text analysis and deep learning: A network approach
• TextConvoNet:A Convolutional Neural Network based Architecture for Text Classification
• Advancements in Natural Language Processing: Exploring Transformer-Based Architectures for Text Understanding