Why Advanced Encoders Lag on Sparse Retrieval? The Answer and an Approach to Bridging Vocabulary Gaps

2026/04/20 by Zhichao Geng, Yang Yang · 1 voice
Computer Science · #Bridging (networking) #Decoding methods #Domain Adaptation and Few-Shot Learning #Encoder #Generative Adversarial Networks and Image Synthesis #Granularity #Intuition #Lag #Lossless compression #Multimodal Machine Learning Applications #Vocabulary #cs.AI #cs.IR #cs.LG

paper · pdf · doi:10.1145/3805712.3809724

arxiv published 2026/04/20 · arxiv updated 2026/04/20 · openalex created_date 2026/07/04 · openalex publication_date 2026/07/10 · openalex updated_date 2026/08/01

Abstract

While advanced foundation models like ModernBERT significantly outperform older architectures in dense retrieval, they surprisingly lag behind the aging BERT-base baseline in learned sparse retrieval (LSR). We identify the root cause as the Vocabulary Gap : modern tokenizers utilize raw, case-sensitive vocabularies designed for lossless reconstruction, which map single semantic units to redundant surface forms, wasting model capacity on morphological noise and hindering lexical matching. We formalize this intuition through a theoretical framework, demonstrating that appropriate vocabulary coarse-graining can tighten the generalization bounds by reducing complexity of the hypothesis class, provided that semantic integrity is preserved. To resolve this, we propose Vocabulary Transfer (VT), a model-agnostic framework that migrates advanced encoders to sparse-friendly, normalized vocabularies with minimal computational cost. VT utilizes a novel Semantic Initialization via spatial topology to preserve geometric structure and an Activation Potential Calibration (APC) mechanism to align pre-trained manifolds with sparsity constraints, preventing the dead neuron and dense collapse observed in standard fine-tuning. Empirically, VT is universally effective: it enables ModernBERT to achieve state-of-the-art performance on the BEIR benchmark (52.4 nDCG, a +4.7 improvement), resuscitates failing models like RoBERTa-large, and generalizes seamlessly to inference-free architectures and specialized domains. These results confirm that the performance lag is not an architectural deficiency but a solvable vocabulary mismatch. We've released our code and models. https://anonymous.4open.science/r/vocab-transfer/. All details included.

Citations

• Milco: Learned Sparse Retrieval Across Languages via a Multilingual Connector
• The Role of Vocabularies in Learning Sparse Representations for Ranking
• Overview of the TREC 2022 deep learning track
• Enhancing Lexicon-Based Text Embeddings with Large Language Models
• Towards Competitive Search Relevance For Inference-Free Learned Sparse Retrievers
• An Empirical Comparison of Vocabulary Expansion and Initialization Approaches for Language Models
• DeeperImpact: Optimizing Sparse Learned Index Structures
• SPLADE-v3: New baselines for SPLADE
• SPRINT: A Unified Toolkit for Evaluating and Demystifying Zero-shot Neural Sparse Retrieval
• From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective
• ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction
• SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking
• A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniques
• Learning Passage Impacts for Inverted Indexes
• BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
• Condenser: a Pre-training Architecture for Dense Retrieval
• COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List
• CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characters
• SparTerm: Learning Term-based Sparse Representation for Fast Text Retrieval
• Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
• Minimizing FLOPs to Learn Efficient Sparse Representations
• Dense Passage Retrieval for Open-Domain Question Answering
• On the Cross-lingual Transferability of Monolingual Representations
• RoBERTa: A Robustly Optimized BERT Pretraining Approach
• Document Expansion by Query Prediction
• Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates
• MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
• Enriching Word Vectors with Subword Information
• Enriching Word Vectors with Subword Information
• From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification
• Neural Machine Translation of Rare Words with Subword Units
• Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Discussions

• I had to look twice at the benchmarking data on sparse retrieval. It turns out modern tokenizers like the one in ModernBERT actually underperform the old BERT-base baseline because their case-sensitiv [bsky, 0 points, 0 comments]

Related

• Improved Learned Sparse Retrieval with Corpus-Specific Vocabularies
• What Are You Token About? Dense Retrieval as Distributions Over the Vocabulary
• Mistral-SPLADE: LLMs for better Learned Sparse Retrieval
• Making Large Language Models Efficient Dense Retrievers
• CSPLADE: Learned Sparse Retrieval with Causal Language Models