QASE Enhanced PLMs: Improved Control in Text Generation for MRC

2024/02/26 by Lin Ai, Ai, Lin, Zheng Hui +5
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Semantic Web and Ontologies #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2403.04771

openalex publication_date 2024/02/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

To address the challenges of out-of-control generation in generative models for machine reading comprehension (MRC), we introduce the Question-Attended Span Extraction (QASE) module. Integrated during the fine-tuning of pre-trained generative language models (PLMs), QASE enables these PLMs to match SOTA extractive methods and outperform leading LLMs like GPT-4 in MRC tasks, without significant increases in computational costs.

Related

• Enhancing Pre-Trained Generative Language Models with Question Attended Span Extraction on Machine Reading Comprehension
• Bridging the Gap between Language Model and Reading Comprehension: Unsupervised MRC via Self-Supervision
• Multi-span Style Extraction for Generative Reading Comprehension
• Composing Answer from Multi-spans for Reading Comprehension
• From Cloze to Comprehension: Retrofitting Pre-trained Masked Language Model to Pre-trained Machine Reader