vix.ing · top · new · best · stats · spec

factgenie: A Framework for Span-based Evaluation of Generated Texts

2024/07/25 by Zdeněk Kasner, Kasner, Zdeněk, Ondřej Plátek +7
Computer Science · #Natural Language Processing Techniques #Semantic Web and Ontologies

paper · pdf · doi:10.48550/arxiv.2407.17863

Abstract

We present factgenie: a framework for annotating and visualizing word spans in textual model outputs. Annotations can capture various span-based phenomena such as semantic inaccuracies or irrelevant text. With factgenie, the annotations can be collected both from human crowdworkers and large language models. Our framework consists of a web interface for data visualization and gathering text annotations, powered by an easily extensible codebase.

Related