Simplicity Level Estimate (SLE): A Learned Reference-Less Metric for Sentence Simplification

2023/10/12 by Cripwell, Liam, Legrand, Joël, Gardent, Claire · 2 citations
#Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2310.08170

Abstract

Automatic evaluation for sentence simplification remains a challenging problem. Most popular evaluation metrics require multiple high-quality references -- something not readily available for simplification -- which makes it difficult to test performance on unseen domains. Furthermore, most existing metrics conflate simplicity with correlated attributes such as fluency or meaning preservation. We propose a new learned evaluation metric (SLE) which focuses on simplicity, outperforming almost all existing metrics in terms of correlation with human judgements.

Cited by

• Text Adaptation to Plain Language and Easy Read via Automatic Post-Editing Cycles
• Evaluating the Effectiveness of Direct Preference Optimization for Personalizing German Automatic Text Simplifications for Persons with Intellectual Disabilities

Related

• Rethinking Automatic Evaluation in Sentence Simplification
• Readability-based Sentence Ranking for Evaluating Text Simplification
• EASSE: Easier Automatic Sentence Simplification Evaluation
• An In-depth Evaluation of Large Language Models in Sentence Simplification with Error-based Human Assessment
• Evaluation Under Imperfect Benchmarks and Ratings: A Case Study in Text Simplification