Towards Minimal Supervision BERT-based Grammar Error Correction

2020/01/10 by Li, Yiyuan, Anastasopoulos, Antonios, Black, Alan W
#Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2001.03521

Abstract

Current grammatical error correction (GEC) models typically consider the task as sequence generation, which requires large amounts of annotated data and limit the applications in data-limited settings. We try to incorporate contextual information from pre-trained language model to leverage annotation and benefit multilingual scenarios. Results show strong potential of Bidirectional Encoder Representations from Transformers (BERT) in grammatical error correction task.

Related

• Stronger Baselines for Grammatical Error Correction Using Pretrained Encoder-Decoder Model
• Weakly Supervised Grammatical Error Correction using Iterative Decoding
• Enhancing Grammatical Error Detection using BERT with Cleaned Lang-8 Dataset
• The Unreasonable Effectiveness of Transformer Language Models in Grammatical Error Correction
• Controllable Data Synthesis Method for Grammatical Error Correction