ChID: A Large-scale Chinese IDiom Dataset for Cloze Test

2019/06/04 by Chujie Zheng, Minlie Huang, Zheng, Chujie +3 · 6 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling #Text Readability and Simplification

paper · pdf · doi:10.48550/arxiv.1906.01265

Abstract

Cloze-style reading comprehension in Chinese is still limited due to the lack of various corpora. In this paper we propose a large-scale Chinese cloze test dataset ChID, which studies the comprehension of idiom, a unique language phenomenon in Chinese. In this corpus, the idioms in a passage are replaced by blank symbols and the correct answer needs to be chosen from well-designed candidate idioms. We carefully study how the design of candidate idioms and the representation of idioms affect the performance of state-of-the-art models. Results show that the machine accuracy is substantially worse than that of human, indicating a large space for further research.

Citations

• Bidirectional Attention Flow for Machine Comprehension

Cited by

• Reasoning or Memorization: Can LLMs Understand and Generate Chinese Xiehouyu Riddles?
• A Set of Quebec-French Corpus of Regional Expressions and Terms
• Are Large Language Models Chronically Online Surfers? A Dataset for Chinese Internet Meme Explanation
• GradMAP: Faster Layer Pruning with Gradient Metric and Projection Compensation
• A Survey of Idiom Datasets for Psycholinguistic and Computational Research
• Evaluating LLMs on Chinese Idiom Translation

Related

• Chengyu-Bench: Benchmarking Large Language Models for Chinese Idiom Understanding and Use
• Chinese Idiom Paraphrasing
• InDEX: Indonesian Idiom and Expression Dataset for Cloze Test
• Synonym Knowledge Enhanced Reader for Chinese Idiom Reading Comprehension
• A BERT-based Dual Embedding Model for Chinese Idiom Prediction