2024/07/01 by Jingheng Ye, Zishan Xu, Ye, Jingheng +17
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Mathematics, Computing, and Information Processing #Natural Language Processing Techniques #Text Readability and Simplification
paper · pdf · doi:10.48550/arxiv.2407.00934
openalex publication_date 2024/07/01 · openalex created_date 2024/07/06 · openalex updated_date 2026/07/28
The paper focuses on the interpretability of Grammatical Error Correction (GEC) evaluation metrics, which received little attention in previous studies. To bridge the gap, we introduce **CLEME2.0**, a reference-based metric describing four fundamental aspects of GEC systems: hit-correction, wrong-correction, under-correction, and over-correction. They collectively contribute to exposing critical qualities and locating drawbacks of GEC systems. Evaluating systems by combining these aspects also leads to superior human consistency over other reference-based and reference-less metrics. Extensive experiments on two human judgment datasets and six reference datasets demonstrate the effectiveness and robustness of our method, achieving a new state-of-the-art result. Our codes are released at https://github.com/THUKElab/CLEME.