Can we repurpose multiple-choice question-answering models to rerank retrieved documents?

2025/03/06 by Jasper Kyle Catapang, Catapang, Jasper Kyle
Computer Science · #Expert finding and Q&A systems #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Information Retrieval and Search Behavior #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2504.06276

openalex publication_date 2025/03/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Yes, repurposing multiple-choice question-answering (MCQA) models for document reranking is both feasible and valuable. This preliminary work is founded on mathematical parallels between MCQA decision-making and cross-encoder semantic relevance assessments, leading to the development of R*, a proof-of-concept model that harmonizes these approaches. Designed to assess document relevance with depth and precision, R* showcases how MCQA's principles can improve reranking in information retrieval (IR) and retrieval-augmented generation (RAG) systems -- ultimately enhancing search and dialogue in AI-powered systems. Through experimental validation, R* proves to improve retrieval accuracy and contribute to the field's advancement by demonstrating a practical prototype of MCQA for reranking by keeping it lightweight.

Related

• JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking
• ASRank: Zero-Shot Re-Ranking with Answer Scent for Document Retrieval
• SciRerankBench: Benchmarking Rerankers Towards Scientific Retrieval-Augmented Generated LLMs
• Enhancing Q&A Text Retrieval with Ranking Models: Benchmarking, fine-tuning and deploying Rerankers for RAG
• Rank-R1: Enhancing Reasoning in LLM-based Document Rerankers via Reinforcement Learning