Deep Learning Based Text Classification: A Comprehensive Review

2020/04/06 by Shervin Minaee, Nal Kalchbrenner, Minaee, Shervin +9 · 2 voices · 11 citations
Computer Science · Mathematics · #Advanced Text Analysis Techniques #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Sentiment Analysis and Opinion Mining #Topic Modeling #cs.CL #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2004.03705

openalex publication_date 2020/04/06 · arxiv published 2020/04/06 · arxiv created 2021/01/04 · arxiv updated 2021/01/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Deep learning based models have surpassed classical machine learning based approaches in various text classification tasks, including sentiment analysis, news categorization, question answering, and natural language inference. In this paper, we provide a comprehensive review of more than 150 deep learning based models for text classification developed in recent years, and discuss their technical contributions, similarities, and strengths. We also provide a summary of more than 40 popular datasets widely used for text classification. Finally, we provide a quantitative analysis of the performance of different deep learning models on popular benchmarks, and discuss future research directions.

Citations

• Training Very Deep Networks
• Skip-Thought Vectors
• Character-Aware Neural Language Models
• Auto-Encoding Variational Bayes
• Gradient-based learning applied to document recognition
• Distributed Representations of Sentences and Documents
• ImageNet classification with deep convolutional neural networks
• Semi-supervised Sequence Learning
• LSTM-based Deep Learning Models for Non-factoid Answer Selection
• Efficient Character-level Document Classification by Combining Convolution and Recurrent Layers
• Attentive Pooling Networks
• A C-LSTM Neural Network for Text Classification
• Modelling Interaction of Sentence Pair with coupled-LSTMs
• Recurrent Neural Network for Text Classification with Multi-Task Learning
• Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention
• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• Semi-Supervised Classification with Graph Convolutional Networks
• Adversarial Training Methods for Semi-Supervised Text Classification
• A Structured Self-attentive Sentence Embedding
• Convolutional Sequence to Sequence Learning
• CondenseNet: An Efficient DenseNet using Learned Group Convolutions
• Stochastic Answer Networks for Machine Reading Comprehension
• Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling
• Investigating Capsule Networks with Dynamic Routing for Text Classification
• Joint Embedding of Words and Labels for Text Classification
• Multi-Task Deep Neural Networks for Natural Language Understanding
• Simplifying Graph Convolutional Networks
• Distilling Task-Specific Knowledge from BERT into Simple Neural Networks
• ERNIE: Enhanced Representation through Knowledge Integration
• A Decomposable Attention Model for Natural Language Inference
• SentiHood: Targeted Aspect Based Sentiment Analysis Dataset for Urban Neighbourhoods
• A large annotated corpus for learning natural language inference
• Unsupervised Data Augmentation for Consistency Training
• A Deep Architecture for Semantic Matching with Multiple Positional Sentence Representations
• A Sensitivity Analysis of (and Practitioners' Guide to) Convolutional Neural Networks for Sentence Classification
• Medical Text Classification using Convolutional Neural Networks
• Neural Attentive Bag-of-Entities Model for Text Classification
• Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
• LLM-Supported Manufacturing Mapping Generation
• Adversarial NLI: A New Benchmark for Natural Language Understanding
• ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
• The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence
• UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training
• MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
• Adversarial Training for Large Neural Language Models
• Language Models are Few-Shot Learners
• GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
• Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
• Robust Conversational AI with Grounded Text Generation
• A Comprehensive Survey on Graph Neural Networks

Cited by

• Convolutional Lie Operator for Sentence Classification
• Evidence-Driven Decision Support for AI Model Selection in Research Software Engineering
• Exploiting the Randomness of Large Language Models (LLM) in Text Classification Tasks: Locating Privileged Documents in Legal Matters
• Optimally Deep Networks -- Adapting Model Depth to Datasets for Superior Efficiency
• Self-supervised Regularization for Text Classification
• Entity Linking Meets Deep Learning: Techniques and Solutions
• Does a Hybrid Neural Network based Feature Selection Model Improve Text Classification?
• Transformer-F: A Transformer network with effective methods for learning universal sentence representation
• Advancing Scientific Text Classification: Fine-Tuned Models with Dataset Expansion and Hard-Voting
• BERTERS: Multimodal representation learning for expert recommendation system with transformers and graph embeddings
• Multimodal Price Prediction
• Justifying AI regulation: Examining multi-stakeholder responses to the AI Act
• MicroNN: An On-device Disk-resident Updatable Vector Database

Discussions

• Deep Learning Based Text Classification: A Comprehensive Review [hn, 1 points, 0 comments]
• Deep Learning Based Text Classification: A Comprehensive Review [hn, 1 points, 0 comments]

Related

• A Survey on Text Classification: From Shallow to Deep Learning
• Text Classification: A Perspective of Deep Learning Methods
• A Comparison of Neural Network Training Methods for Text Classification
• Text Classification: Neural Networks VS Machine Learning Models VS Pre-trained Models
• Classifying textual data: shallow, deep and ensemble methods