An Empirical Evaluation of Probabilistic Lexicalized Tree Insertion Grammars

1998/08/04 by Rebecca Hwa
Computer Science · #cmp-lg #cs.CL

paper · pdf

published as Proceedings of COLING-ACL'98 · 10 pages, 6 encapsulated postscript figures and 2 latex figures, uses colacl.sty

arxiv created 1998/08/04 · arxiv updated 2009/11/30

Abstract

We present an empirical study of the applicability of Probabilistic Lexicalized Tree Insertion Grammars (PLTIG), a lexicalized counterpart to Probabilistic Context-Free Grammars (PCFG), to problems in stochastic natural-language processing. Comparing the performance of PLTIGs with non-hierarchical N-gram models and PCFGs, we show that PLTIG combines the best aspects of both, with language modeling capability comparable to N-grams, and improved parsing performance over its non-lexicalized counterpart. Furthermore, training of PLTIGs displays faster convergence than PCFGs.

Related

• Building Probabilistic Models for Natural Language
• PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols
• Three Generative, Lexicalised Models for Statistical Parsing
• Prefix Probabilities from Stochastic Tree Adjoining Grammars
• Parsing with the Shortest Derivation