Word Segmentation as Graph Partition

2018/04/05 by Yuanhao Liu, Liu, Yuanhao, Sheng Yu +1
Computer Science · #Algorithms and Data Compression #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling #cs.CL

paper · pdf · doi:10.48550/arxiv.1804.01778

arxiv created 2018/04/05 · openalex publication_date 2018/04/05 · arxiv updated 2018/04/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose a new approach to the Chinese word segmentation problem that considers the sentence as an undirected graph, whose nodes are the characters. One can use various techniques to compute the edge weights that measure the connection strength between characters. Spectral graph partition algorithms are used to group the characters and achieve word segmentation. We follow the graph partition approach and design several unsupervised algorithms, and we show their inspiring segmentation results on two corpora: (1) electronic health records in Chinese, and (2) benchmark data from the Second International Chinese Word Segmentation Bakeoff.

Citations

• A tutorial on hidden Markov models and selected applications in speech recognition
• A tutorial on spectral clustering

Related

• Chinese Word Segmentation with Heterogeneous Graph Neural Network
• A New Clustering neural network for Chinese word segmentation
• Parsing Through Boundaries in Chinese Word Segmentation
• A Graph-based Model for Joint Chinese Word Segmentation and Dependency Parsing
• A realistic and robust model for Chinese word segmentation