2024/10/22 by Gefen Dawidowicz, Dawidowicz, Gefen, Elad Hirsch +3 · 2 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · Medicine · #AI in cancer detection #Biomedical Text Mining and Ontologies #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Radiomics and Machine Learning in Medical Imaging
paper · pdf · doi:10.48550/arxiv.2410.17357
openalex publication_date 2024/10/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The paper proposes a novel evaluation metric for automatic medical report generation from X-ray images, VLScore. It aims to overcome the limitations of existing evaluation methods, which either focus solely on textual similarities, ignoring clinical aspects, or concentrate only on a single clinical aspect, the pathology, neglecting all other factors. The key idea of our metric is to measure the similarity between radiology reports while considering the corresponding image. We demonstrate the benefit of our metric through evaluation on a dataset where radiologists marked errors in pairs of reports, showing notable alignment with radiologists' judgments. In addition, we provide a new dataset for evaluating metrics. This dataset includes well-designed perturbations that distinguish between significant modifications (e.g., removal of a diagnosis) and insignificant ones. It highlights the weaknesses in current evaluation metrics and provides a clear framework for analysis.