Attention Is All You Need for Chinese Word Segmentation

2019/10/31 by Sufeng Duan, Hai Zhao, Duan, Sufeng +1
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1910.14537

openalex publication_date 2019/10/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Taking greedy decoding algorithm as it should be, this work focuses on further strengthening the model itself for Chinese word segmentation (CWS), which results in an even more fast and more accurate CWS model. Our model consists of an attention only stacked encoder and a light enough decoder for the greedy segmentation plus two highway connections for smoother training, in which the encoder is composed of a newly proposed Transformer variant, Gaussian-masked Directional (GD) Transformer, and a biaffine attention scorer. With the effective encoder design, our model only needs to take unigram features for scoring. Our model is evaluated on SIGHAN Bakeoff benchmark datasets. The experimental results show that with the highest segmentation speed, the proposed model achieves new state-of-the-art or comparable performance against strong baselines in terms of strict closed test setting.

Citations

• Efficient Estimation of Word Representations in Vector Space
• Chinese Word Segmentation: Another Decade Review (2007-2017)
• Toward Fast and Accurate Neural Chinese Word Segmentation with Multi-Criteria Learning

Related

• Green CWS: Extreme Distillation and Efficient Decode Method Towards Industrial Application
• A New Clustering neural network for Chinese word segmentation
• RethinkCWS: Is Chinese Word Segmentation a Solved Task?
• BERT Meets Chinese Word Segmentation
• A realistic and robust model for Chinese word segmentation