2021/04/09 by Francesco Moramarco, Damir Jurić, Moramarco, Francesco +5 · 1 citation
Biochemistry, Genetics and Molecular Biology · Computer Science · #Biomedical Text Mining and Ontologies #Computation and Language (cs.CL) #FOS: Computer and information sciences #Semantic Web and Ontologies #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2104.04412
openalex publication_date 2021/04/09 · openalex created_date 2021/04/26 · openalex updated_date 2026/07/28
We propose a method for evaluating the quality of generated text by asking evaluators to count facts, and computing precision, recall, f-score, and accuracy from the raw counts. We believe this approach leads to a more objective and easier to reproduce evaluation. We apply this to the task of medical report summarisation, where measuring objective quality and accuracy is of paramount importance.