2019/03/19 by Graham Neubig, Zi-Yi Dou, Neubig, Graham +11 · 4 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Speech and dialogue systems #Topic Modeling
paper · pdf · doi:10.48550/arxiv.1903.07926
openalex publication_date 2019/03/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In this paper, we describe compare-mt, a tool for holistic analysis and\ncomparison of the results of systems for language generation tasks such as\nmachine translation. The main goal of the tool is to give the user a high-level\nand coherent view of the salient differences between systems that can then be\nused to guide further analysis or system improvement. It implements a number of\ntools to do so, such as analysis of accuracy of generation of particular types\nof words, bucketed histograms of sentence accuracies or counts based on salient\ncharacteristics, and extraction of characteristic n-grams for each system. It\nalso has a number of advanced features such as use of linguistic labels, source\nside data, or comparison of log likelihoods for probabilistic models, and also\naims to be easily extensible by users to new types of analysis. The code is\navailable at https://github.com/neulab/compare-mt\n