Pretrained Transformers for Simple Question Answering over Knowledge Graphs

2020/01/31 by Denis Lukovnikov, D. Lukovnikov, Asja Fischer +6
Computer Science · Engineering · #Advanced Graph Neural Networks #Artificial intelligence #Computer science #Convolutional neural network #Engineering #Knowledge graph #Language model #Machine learning #Natural Language Processing Techniques #Natural language #Natural language processing #Question answering #Simple (philosophy) #Task (project management) #Topic Modeling #Transformer #cs.AI #cs.CL #cs.LG

paper · pdf · doi:10.48550/arxiv.2001.11985

published in arXiv (Cornell University) (Cornell University)

arxiv created 2020/01/31 · openalex publication_date 2020/01/31 · arxiv updated 2020/02/03 · openalex created_date 2020/02/07 · openalex updated_date 2026/08/05

Abstract

Answering simple questions over knowledge graphs is a well-studied problem in question answering. Previous approaches for this task built on recurrent and convolutional neural network based architectures that use pretrained word embeddings. It was recently shown that finetuning pretrained transformer networks (e.g. BERT) can outperform previous approaches on various natural language processing tasks. In this work, we investigate how well BERT performs on SimpleQuestions and provide an evaluation of both BERT and BiLSTM-based models in datasparse scenarios.

Citations

• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Related

• An Empirical Study of Pre-trained Language Models in Simple Knowledge Graph Question Answering
• Structured Pruning of a BERT-based Question Answering Model
• Span Selection Pre-training for Question Answering
• Exploring Neural Net Augmentation to BERT for Question Answering on SQUAD 2.0
• Syntax-informed Question Answering with Heterogeneous Graph Transformer