Boosting Data Utilization for Multilingual Dense Retrieval

2025/09/11 by Huang, Chao, Mo, Fengran, Chen, Yufeng +5 · 1 citation
#FOS: Computer and information sciences #Information Retrieval (cs.IR)

paper · doi:10.48550/arxiv.2509.09459

Abstract

Multilingual dense retrieval aims to retrieve relevant documents across different languages based on a unified retriever model. The challenge lies in aligning representations of different languages in a shared vector space. The common practice is to fine-tune the dense retriever via contrastive learning, whose effectiveness highly relies on the quality of the negative sample and the efficacy of mini-batch data. Different from the existing studies that focus on developing sophisticated model architecture, we propose a method to boost data utilization for multilingual dense retrieval by obtaining high-quality hard negative samples and effective mini-batch data. The extensive experimental results on a multilingual retrieval benchmark, MIRACL, with 16 languages demonstrate the effectiveness of our method by outperforming several existing strong baselines.

Citations

• ConvMix: A Mixed-Criteria Data Augmentation Framework for Conversational Dense Retrieval
• UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversations
• A Survey of Model Architectures in Information Retrieval
• SyNeg: LLM-Driven Synthetic Hard-Negatives for Dense Retrieval
• Conan-embedding: General Text Embedding with More and Better Negative Samples
• The Llama 3 Herd of Models
• NV-Retriever: Improving text embedding models with effective hard-negative mining
• NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models
• Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Models
• Unsupervised Multilingual Dense Retrieval via Generative Pseudo Labeling
• M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation
• Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrieval
• Learning to Relate to Previous Turns in Conversational Search
• ConvGQR: Generative Query Reformulation for Conversational Search
• Extending English IR methods to multi-lingual IR
• Text Embeddings by Weakly-Supervised Contrastive Pre-training
• Language Agnostic Multilingual Information Retrieval with Contrastive Learning
• Neural Approaches to Multilingual Information Retrieval
• Disentangled Modeling of Domain and Relevance for Adaptable Dense Retrieval
• Keywords and Instances: A Hierarchical Contrastive Learning Framework Unifying Hybrid Granularities for Text Generation
• Towards Best Practices for Training Multilingual Dense Retrieval Models
• Tevatron: An Efficient and Flexible Toolkit for Dense Retrieval
• Unsupervised Dense Information Retrieval with Contrastive Learning
• One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval
• RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering
• Pretrained Transformers for Text Ranking: BERT and Beyond
• Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval
• MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer
• Don't Stop Pretraining: Adapt Language Models to Domains and Tasks
• Dense Passage Retrieval for Open-Domain Question Answering
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Unsupervised Cross-lingual Representation Learning at Scale
• Multi-Stage Document Ranking with BERT
• HuggingFace's Transformers: State-of-the-art Natural Language Processing
• FastText.zip: Compressing text classification models
• MS MARCO: A Human Generated MAchine Reading COmprehension Dataset
• Bag of Tricks for Efficient Text Classification
• Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challenges
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Cited by

• SoT: Structured-of-Thought Prompting Guides Multilingual Reasoning in Large Language Models

Related

• Unsupervised Multilingual Dense Retrieval via Generative Pseudo Labeling
• Towards Best Practices for Training Multilingual Dense Retrieval Models
• Mr. TyDi: A Multi-lingual Benchmark for Dense Retrieval
• Retrofitting Small Multilingual Models for Retrieval: Matching 7B Performance with 300M Parameters
• Unsupervised Dense Information Retrieval with Contrastive Learning