Training Documents Reranker with Search Rubrics for Deep Research Agent

2026/08/04 by Wenhan Liu, Yu Lu, Qiaolin Xia +8
Computer Science · #cs.IR #cs.AI #cs.CL

paper · pdf

28 pages

arxiv created 2026/08/04 · arxiv updated 2026/08/05

Abstract

Retrieval systems help deep research agents generate high-quality answers by providing relevant documents. However, existing retrievers typically select documents through relevance matching, while individually well-matched top-k documents may not form a set that satisfies the complex information needs of an agent query (\eg, diverse, concise and authoritative documents). In this paper, we propose search-oriented rubrics that explicitly define the requirements that high-quality document sets should satisfy for each agent query. Our search rubrics are organized into a hierarchical structure and synthesized using a powerful LLM. Based on these search rubrics, we further train a document reranker RubricRanker to select a high-quality subset from retrieved documents. We design a two-stage training framework that consists of rubrics-guided supervised fine-tuning and rubric-based reinforcement learning. Extensive experiments demonstrate that RubricRanker outperforms the strongest baseline by 2.6 points on four deep research benchmarks and generalizes well to five RAG benchmarks.

Citations

• DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research
• AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following
• ResearchQA: Evaluating Scholarly Question Answering at Scale Across 75 Fields with Survey-Mined Questions and Rubrics
• ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability
• Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains
• DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents
• Search Arena: Analyzing Search-Augmented LLMs
• HealthBench: Evaluating Large Language Models Towards Improved Human Health
• CoRanking: Collaborative Ranking with Small and Large Ranking Agents
• Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
• WebWalker: Benchmarking LLMs in Web Traversal
• OpenScholar: Synthesizing Scientific Literature with Retrieval-augmented LMs
• DemoRank: Selecting Effective Demonstrations for Large Language Models in Ranking Task
• BioXP-0.5B: Explainable Medical-AI via RL-GRPO
• RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!
• C-Pack: Packed Resources For General Chinese Embeddings
• FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
• Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting
• When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories
• Text Embeddings by Weakly-Supervised Contrastive Pre-training
• RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses
• Measuring and Narrowing the Compositionality Gap in Language Models
• Dense Passage Retrieval for Open-Domain Question Answering
• Document Ranking with a Pretrained Sequence-to-Sequence Model
• HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
• TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for\n Reading Comprehension