Techniques to Improve Q&A Accuracy with Transformer-based models on Large Complex Documents

2020/09/26 by Chejui Liao, Liao, Chejui, Tabish Maniar +5
Computer Science · #Advanced Text Analysis Techniques #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Expert finding and Q&A systems #FOS: Computer and information sciences #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2009.12695

openalex publication_date 2020/09/26 · openalex created_date 2024/04/11 · openalex updated_date 2026/07/28

Abstract

This paper discusses the effectiveness of various text processing techniques, their combinations, and encodings to achieve a reduction of complexity and size in a given text corpus. The simplified text corpus is sent to BERT (or similar transformer based models) for question and answering and can produce more relevant responses to user queries. This paper takes a scientific approach to determine the benefits and effectiveness of various techniques and concludes a best-fit combination that produces a statistically significant improvement in accuracy.

Related

• Delaying Interaction Layers in Transformer-based Encoders for Efficient Open Domain Question Answering
• Structured Pruning of a BERT-based Question Answering Model
• DeFormer: Decomposing Pre-trained Transformers for Faster Question Answering
• FAT ALBERT: Finding Answers in Large Texts using Semantic Similarity Attention Layer based on BERT
• Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges