gec-metrics: A Unified Library for Grammatical Error Correction Evaluation

2025/05/26 by Takumi Goto, Yusuke Sakai, Goto, Takumi +3 · 2 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Text Readability and Simplification

paper · pdf · doi:10.48550/arxiv.2505.19388

openalex publication_date 2025/05/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We introduce gec-metrics, a library for using and developing grammatical error correction (GEC) evaluation metrics through a unified interface. Our library enables fair system comparisons by ensuring that everyone conducts evaluations using a consistent implementation. Moreover, it is designed with a strong focus on API usage, making it highly extensible. It also includes meta-evaluation functionalities and provides analysis and visualization scripts, contributing to developing GEC evaluation metrics. Our code is released under the MIT license and is also distributed as an installable package. The video is available on YouTube.

Cited by

• IndicGEC: Powerful Models, or a Measurement Mirage?
• Reliability Crisis of Reference-free Metrics for Grammatical Error Correction

Related

• CLEME2.0: Towards Interpretable Evaluation by Disentangling Edits for Grammatical Error Correction
• There's No Comparison: Reference-less Evaluation Metrics in Grammatical Error Correction
• Rethinking Evaluation Metrics for Grammatical Error Correction: Why Use a Different Evaluation Process than Human?
• IMPARA-GED: Grammatical Error Detection is Boosting Reference-free Grammatical Error Quality Estimator
• Evaluation of large-scale synthetic data for Grammar Error Correction