Syllable-aware Neural Language Models: A Failure to Beat Character-aware Ones

2017/07/20 by Assylbekov, Zhenisbek, Takhanov, Rustem, Myrzakhmetov, Bagdat +1
#68T50 #Computation and Language (cs.CL) #FOS: Computer and information sciences #I.2.7 #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE)

paper · doi:10.48550/arxiv.1707.06480

Abstract

Syllabification does not seem to improve word-level RNN language modeling quality when compared to character-based segmentation. However, our best syllable-aware language model, achieving performance comparable to the competitive character-aware model, has 18%-33% fewer parameters and is trained 1.2-2.2 times faster.

Related

• Revisiting Neural Language Modelling with Syllables
• Character-Aware Neural Language Models
• Character-Level Language Modeling with Hierarchical Recurrent Neural Networks
• Syllable-level Neural Language Model for Agglutinative Language
• Reusing Weights in Subword-aware Neural Language Models