Typo-Robust Representation Learning for Dense Retrieval

2023/06/17 by Panuthep Tasawong, Wuttikorn Ponwitayarat, Tasawong, Panuthep +9 · 1 citation
Computer Science · #Advanced Image and Video Retrieval Techniques #Computation and Language (cs.CL) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #I.2.7 #Information Retrieval (cs.IR) #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.2306.10348

openalex publication_date 2023/06/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Dense retrieval is a basic building block of information retrieval applications. One of the main challenges of dense retrieval in real-world settings is the handling of queries containing misspelled words. A popular approach for handling misspelled queries is minimizing the representations discrepancy between misspelled queries and their pristine ones. Unlike the existing approaches, which only focus on the alignment between misspelled and pristine queries, our method also improves the contrast between each misspelled query and its surrounding queries. To assess the effectiveness of our proposed method, we compare it against the existing competitors using two benchmark datasets and two base encoders. Our method outperforms the competitors in all cases with misspelled queries. Our code and models are available at https://github. com/panuthept/DST-DenseRetrieval.

Cited by

• Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models

Related

• Typos-aware Bottlenecked Pre-Training for Robust Dense Retrieval
• Improving the Robustness of Dense Retrievers Against Typos via Multi-Positive Contrastive Learning
• Dealing with Typos for BERT-based Passage Retrieval and Ranking
• Noise-Robust Dense Retrieval via Contrastive Alignment Post Training
• QE-RAG: A Robust Retrieval-Augmented Generation Benchmark for Query Entry Errors