Models and Data for Simple Applications of BERT for Ad Hoc Document Retrieval

2019/03/26 by Akkalyoncu Yilmaz, Zeynep, Wei Yang, Yang, Wei +4 · 2 citations
Computer Science · #Computation and Language (cs.CL) #Expert finding and Q&A systems #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Natural Language Processing Techniques #Topic Modeling

paper · pdf · doi:10.48550/arxiv.1903.10972

openalex publication_date 2019/03/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Following recent successes in applying BERT to question answering, we explore simple applications to ad hoc document retrieval. This required confronting the challenge posed by documents that are typically longer than the length of input BERT was designed to handle. We address this issue by applying inference on sentences individually, and then aggregating sentence scores to produce document scores. Experiments on TREC microblog and newswire test collections show that our approach is simple yet effective, as we report the highest average precision on these datasets by neural approaches that we are aware of.

Citations

• Evaluating Sentence-Level Relevance Feedback for High-Recall Information Retrieval
• Passage Re-ranking with BERT

Cited by

• B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval
• An Audio-enriched BERT-based Framework for Spoken Multiple-choice Question Answering
• A Deep Look into Neural Ranking Models for Information Retrieval

Related

• Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges
• DC-BERT: Decoupling Question and Document for Efficient Contextual Encoding
• Co-BERT: A Context-Aware BERT Retrieval Model Incorporating Local and Query-specific Context
• Universal Text Representation from BERT: An Empirical Study
• Understanding the Behaviors of BERT in Ranking