DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruning
2025/09/28 by Yibo Yan, Guangwei Xu, Yan, Yibo +9 · 3 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Computation and Language (cs.CL) #Document retrieval #Embedding #FOS: Computer and information sciences #Image Retrieval and Classification Techniques #Information Retrieval (cs.IR) #Leverage (statistics) #Pruning #Task (project management) #Video Analysis and Summarization
paper · pdf · doi:10.48550/arxiv.2509.23883
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2025/09/28 · openalex created_date 2025/10/19 · openalex updated_date 2026/08/05
Abstract
Visual Document Retrieval (VDR), the task of retrieving visually-rich document pages using queries that combine visual and textual cues, is crucial for numerous real-world applications. Recent state-of-the-art methods leverage Large Vision-Language Models (LVLMs) in a multi-vector paradigm, representing each document as patch-level embeddings to capture fine-grained details. While highly effective, this approach introduces a critical challenge: prohibitive storage overhead, as storing hundreds of vectors per page makes large-scale deployment costly and impractical. To address this, we introduce DocPruner, the first framework to employ adaptive patch-level embedding pruning for VDR to effectively reduce the storage overhead. DocPruner leverages the intra-document patch attention distribution to dynamically identify and discard redundant embeddings for each document. This adaptive mechanism enables a significant 50-60% reduction in storage for leading multi-vector VDR models with negligible degradation in document retrieval performance. Extensive experiments across more than ten representative datasets validate that DocPruner offers a robust, flexible, and effective solution for building storage-efficient, large-scale VDR systems.
Citations
- Vector embedding of multi-modal texts: a tool for discovery?
- Multimodal Data Storage and Retrieval for Embodied AI: A Survey
- VLA-Mark: A cross modal watermark for large vision-language alignment model
- Llama Nemoretriever Colembed: Top-Performing Text-Image Retrieval Model
- VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documents
- Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers
- jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrieval
- Towards Natural Language-Based Document Image Retrieval: New Dataset and Benchmark
- Towards Storage-Efficient Visual Document Retrieval: An Empirical Study on Reducing Patch-Level Embeddings
- ViDoRe Benchmark V2: Raising the Bar for Visual Retrieval
- Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysis
- CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoring
- Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrieval
- A Multi-Granularity Retrieval Framework for Visually-Rich Documents
- VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documents
- SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models
- Efficient Constant-Space Multi-Vector Retrieval
- A Survey of Multimodal Retrieval-Augmented Generation
- MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detection
- Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlook
- Composed Multi-modal Retrieval: A Survey of Approaches and Applications
- A Survey on Mechanistic Interpretability for Multi-Modal Foundation Models
- Qwen2.5-VL Technical Report
- SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearning
- Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?
- Any Information Is Just Worth One Single Screenshot: Unifying Search With Visualized Information Retrieval
- MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models
- Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligence
- Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation
- Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning
- WARP: An Efficient Engine for Multi-Vector Retrieval
- MMDocIR: Benchmarking Multimodal Retrieval for Long Documents
- Instruction-Following Pruning for Large Language Models
- GME: Improving Universal Multimodal Retrieval by Multimodal LLMs
- A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges
- OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generation
- Code Artifact for Efficient LLM Inference using Dynamic Input Pruning and Cache-Aware Masking
- Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMs
- ATP-LLaVA: Adaptive Token Pruning for Large Vision Language Models
- 2D Matryoshka Training for Information Retrieval
- MM-Embed: Universal Multimodal Retrieval with Multimodal LLMs
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks
- Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality
- ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detection
- SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference
- Reducing the Footprint of Multi-Vector Retrieval with Minimal Performance Impact via Token Pooling
- Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Models
- mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understanding
- GeoReasoner: Reasoning On Geospatially Grounded Context For Natural Language Understanding
- Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models
- LLaVA-OneVision: Easy Visual Task Transfer
- Deep Learning based Visually Rich Document Content Understanding: A Survey
- The Llama 3 Herd of Models
- Matryoshka-Adaptor: Unsupervised and Supervised Tuning for Smaller Embedding Dimensions
- PaliGemma: A versatile 3B VLM for transfer
- LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models
- ColPali: Efficient Document Retrieval with Vision Language Models
- DocKylin: A Large Multimodal Model for Visual Document Understanding with Efficient Visual Slimming
- Efficient Document Ranking with Learnable Late Interactions
- FastMem: Fast Memorization of Prompt Improves Context Awareness of Large Language Models
- Unifying Multimodal Retrieval via Document Screenshot Embedding
- MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Model
- MUVERA: Multi-Vector Retrieval via Fixed Dimensional Encodings
- xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token
- Hallucination of Multimodal Large Language Models: A Survey
- UrbanCross: Enhancing Satellite Image-Text Retrieval with Cross-Domain Adaptation
- Generative Retrieval as Multi-Vector Dense Retrieval
- FIT-RAG: Black-Box RAG with Factual Information and Token Reduction
- mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understanding
- An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models
- Enhancing Visual Document Understanding with Contrastive Learning in Large Visual-Language Models
- Large Multimodal Agents: A Survey
- The Revolution of Multimodal Large Language Models: A Survey
- RAP: Retrieval-Augmented Planning with Contextual Memory for Multimodal LLM Agents
- M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation
- Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning
- Agent AI: Surveying the Horizons of Multimodal Interaction
- Improving Text Embeddings with Large Language Models
- ESPN: Memory-Efficient Multi-Vector Information Retrieval
- TCRA-LLM: Token Compression Retrieval Augmented Large Language Model for Inference Cost Reduction
- UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Web
- C-Pack: Packed Resources For General Chinese Embeddings
- Cross-Modal Retrieval: A Systematic Review of Methods and Future Directions
- A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations
- A Survey on Deep Neural Network Pruning: Taxonomy, Comparison, Analysis, and Recommendations
- A Static Pruning Study on Sparse Neural Retrievers
- Rethinking the Role of Token Retrieval in Multi-Vector Retrieval
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
- VQA and Visual Reasoning: An Overview of Recent Datasets, Methods and Challenges
- Token Merging: Your ViT But Faster
- PLAID: An Efficient Engine for Late Interaction Retrieval
- ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction
- Whitening Sentence Representations for Better Semantics and Faster Retrieval
- ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
- Dense Passage Retrieval for Open-Domain Question Answering
- Deep Learning and the Information Bottleneck Principle
- The information bottleneck method
Cited by
Related