CoRECT: A Framework for Evaluating Embedding Compression Techniques at Scale

2025/10/22 by Laura Caspari, Caspari, L., Michael Dinzinger +9
Computer Science · #Advanced Image and Video Retrieval Techniques #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Information Retrieval and Search Behavior #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2510.19340

openalex publication_date 2025/10/22 · openalex created_date 2025/10/24 · openalex updated_date 2026/07/28

Abstract

Dense retrieval systems have proven to be effective across various benchmarks, but require substantial memory to store large search indices. Recent advances in embedding compression show that index sizes can be greatly reduced with minimal loss in ranking quality. However, existing studies often overlook the role of corpus complexity -- a critical factor, as recent work shows that both corpus size and document length strongly affect dense retrieval performance. In this paper, we introduce CoRECT (Controlled Retrieval Evaluation of Compression Techniques), a framework for large-scale evaluation of embedding compression methods, supported by a newly curated dataset collection. To demonstrate its utility, we benchmark eight representative types of compression methods. Notably, we show that non-learned compression achieves substantial index size reduction, even on up to 100M passages, with statistically insignificant performance loss. However, selecting the optimal compression method remains challenging, as performance varies across models. Such variability highlights the necessity of CoRECT to enable consistent comparison and informed selection of compression methods. All code, data, and results are available on GitHub and HuggingFace.

Citations

• jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrieval
• Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models
• Gemini Embedding: Generalizable Embeddings from Gemini
• Arctic-Embed 2.0: Multilingual Retrieval Without Compromise
• jina-embeddings-v3: Multilingual Embeddings With Task LoRA
• Matryoshka-Adaptor: Unsupervised and Supervised Tuning for Smaller Embedding Dimensions
• Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Models
• Multilingual E5 Text Embeddings: A Technical Report
• Dense X Retrieval: What Retrieval Granularity Should We Use?
• Experimental Analysis of Large-scale Learnable Vector Storage Compression
• Matryoshka Representation Learning
• Injecting Domain Adaptation with Learning-to-hash for Effective and Efficient Zero-shot Dense Retrieval
• Distill-VQ: Learning Retrieval Oriented Vector Quantization By Distilling Knowledge from Dense Embeddings
• Jointly Optimizing Query Encoder and Product Quantization to Improve Retrieval Performance
• Efficient Passage Retrieval with Hashing for Open-domain Question Answering
• BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models
• The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizes
• Sparse, Dense, and Attentional Representations for Text Retrieval
• Dense Passage Retrieval for Open-Domain Question Answering
• PyTorch: An Imperative Style, High-Performance Deep Learning Library

Related

• The Future is Sparse: Embedding Compression for Scalable Retrieval in Recommender Systems
• Dimension Reduction for Efficient Dense Retrieval via Conditional Autoencoder
• Experimental Analysis of Large-scale Learnable Vector Storage Compression
• CORE-RAG: Lossless Compression for Retrieval-Augmented LLMs via Reinforcement Learning
• Compressibility of Distributed Document Representations