A Survey on Neural Network Language Models

2019/06/09 by Kun Jing, Jing, Kun, Jungang Xu +1
Computer Science · #Advanced Computational Techniques and Applications #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1906.03591

openalex publication_date 2019/06/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

As the core component of Natural Language Processing (NLP) system, Language Model (LM) can provide word representation and probability indication of word sequences. Neural Network Language Models (NNLMs) overcome the curse of dimensionality and improve the performance of traditional LMs. A survey on NNLMs is performed in this paper. The structure of classic NNLMs is described firstly, and then some major improvements are introduced and analyzed. We summarize and compare corpora and toolkits of NNLMs. Further, some research directions of NNLMs are discussed.

Citations

• Efficient Estimation of Word Representations in Vector Space
• Larger-Context Language Modelling
• Gated Word-Character Recurrent Language Model

Related

• A Study on Neural Network Language Modeling
• A Primer on Neural Network Models for Natural Language Processing
• Analysis Methods in Neural Language Processing: A Survey
• A Survey of the Usages of Deep Learning in Natural Language Processing
• An Analysis of Neural Language Modeling at Multiple Scales