MiLQ: Benchmarking IR Models for Bilingual Web Search with Mixed Language Queries

2025/05/22 by Jonghwi Kim, Deokhyung Kang, Kim, Jonghwi +9
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Information Retrieval and Search Behavior #Natural Language Processing Techniques #Web Data Mining and Analysis

paper · pdf · doi:10.48550/arxiv.2505.16631

openalex publication_date 2025/05/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Despite bilingual speakers frequently using mixed-language queries in web searches, Information Retrieval (IR) research on them remains scarce. To address this, we introduce MiLQ, Mixed-Language Query test set, the first public benchmark of mixed-language queries, qualified as realistic and relatively preferred. Experiments show that multilingual IR models perform moderately on MiLQ and inconsistently across native, English, and mixed-language queries, also suggesting code-switched training data's potential for robust IR models handling such queries. Meanwhile, intentional English mixing in queries proves an effective strategy for bilinguals searching English documents, which our analysis attributes to enhanced token matching compared to native queries.

Citations

• Cross-Dialect Information Retrieval: Information Access in Low-Resource and High-Variance Languages
• Translate-Distill: Learning Cross-Language Dense Retrieval by Translation and Distillation
• Boosting Zero-shot Cross-lingual Retrieval by Training on Artificially Code-Switched Data
• Overview of the TREC 2022 NeuCLIR Track
• Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages
• Transfer Learning Approaches for Building Cross-Language Dense Retrieval Models
• Unsupervised Dense Information Retrieval with Contrastive Learning
• Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representations
• XOR QA: Cross-lingual Open-Retrieval Question Answering
• ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
• A Survey of Code-switched Speech and Language Processing
• Word Translation Without Parallel Data
• FastText.zip: Compressing text classification models
• MS MARCO: A Human Generated MAchine Reading COmprehension Dataset

Related

• Bridging Language Gaps: Advances in Cross-Lingual Information Retrieval with Multilingual LLMs
• Boosting Zero-shot Cross-lingual Retrieval by Training on Artificially Code-Switched Data
• Improving Korean-English Cross-Lingual Retrieval: A Data-Centric Study of Language Composition and Model Merging
• mFollowIR: a Multilingual Benchmark for Instruction Following in Retrieval
• What Drives Cross-lingual Ranking? Retrieval Approaches with Multilingual Language Models