Making Neural Machine Reading Comprehension Faster

2019/03/29 by Debajyoti Chatterjee, Chatterjee, Debajyoti
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1904.00796

openalex publication_date 2019/03/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This study aims at solving the Machine Reading Comprehension problem where questions have to be answered given a context passage. The challenge is to develop a computationally faster model which will have improved inference time. State of the art in many natural language understanding tasks, BERT model, has been used and knowledge distillation method has been applied to train two smaller models. The developed models are compared with other models which have been developed with the same intention.

Citations

• Adam: A Method for Stochastic Optimization
• Distilling the Knowledge in a Neural Network
• Words or Characters? Fine-grained Gating for Reading Comprehension
• Learning to Compute Word Embeddings On the Fly
• Stochastic Answer Networks for Machine Reading Comprehension
• Lightweight Convolutional Approaches to Reading Comprehension on SQuAD
• Structural Embedding of Syntactic Trees for Machine Comprehension
• Xception: Deep Learning with Depthwise Separable Convolutions
• DTMT: A Novel Deep Transition Architecture for Neural Machine Translation
• Layer Normalization

Related

• A Survey on Neural Machine Reading Comprehension
• A Study of the Tasks and Models in Machine Reading Comprehension
• Machine Reading, Fast and Slow: When Do Models "Understand" Language?
• Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond
• Building Large Machine Reading-Comprehension Datasets using Paragraph Vectors