2020/11/08 by Craig Thomson, Ehud Reiter, Thomson, Craig +1 · 1 citation
Computer Science · #Natural Language Processing Techniques #Semantic Web and Ontologies #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2011.03992
Most Natural Language Generation systems need to produce accurate texts. We\npropose a methodology for high-quality human evaluation of the accuracy of\ngenerated texts, which is intended to serve as a gold-standard for accuracy\nevaluations of data-to-text systems. We use our methodology to evaluate the\naccuracy of computer generated basketball summaries. We then show how our gold\nstandard evaluation can be used to validate automated metrics\n