2001/10/11 by Andre Kempe
Computer Science · #cs.CL
published as Proc. CLIN 2000, pp. 88-96, Tilburg, The Netherlands. November 3 · 7 pages, 4 figures, LaTeX (+ eps)
arxiv created 2001/10/11 · arxiv updated 2009/11/30
We present a method of constructing and using a cascade consisting of a left- and a right-sequential finite-state transducer (FST), T1 and T2, for part-of-speech (POS) disambiguation. Compared to an HMM, this FST cascade has the advantage of significantly higher processing speed, but at the cost of slightly lower accuracy. Applications such as Information Retrieval, where the speed can be more important than accuracy, could benefit from this approach. In the process of tagging, we first assign every word a unique ambiguity class ci that can be looked up in a lexicon encoded by a sequential FST. Every ci is denoted by a single symbol, e.g. [ADJNOUN], although it represents a set of alternative tags that a given word can occur with. The sequence of the ci of all words of one sentence is the input to our FST cascade. It is mapped by T1, from left to right, to a sequence of reduced ambiguity classes ri. Every ri is denoted by a single symbol, although it represents a set of alternative tags. Intuitively, T1 eliminates the less likely tags from ci, thus creating ri. Finally, T2 maps the sequence of ri, from right to left, to a sequence of single POS tags ti. Intuitively, T2 selects the most likely ti from every ri. The probabilities of all ti, ri, and ci are used only at compile time, not at run time. They do not (directly) occur in the FSTs, but are "implicitly contained" in their structure.