2025/06/26 by Badri Vishal Kasuba, Kasuba, Badri Vishal, Parag Chaudhuri +3
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Granularity #Ground #Interpretability #Matching (statistics) #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Point (geometry) #Question answering #Set (abstract data type) #Video Analysis and Summarization
paper · pdf · doi:10.48550/arxiv.2506.21316
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2025/06/26 · openalex created_date 2025/10/14 · openalex updated_date 2026/08/05
Visual grounding in text-rich document images is a critical yet underexplored challenge for Document Intelligence and Visual Question Answering (VQA) systems. We present DRISHTIKON, a multi-granular and multi-block visual grounding framework designed to enhance interpretability and trust in VQA for complex, multilingual documents. Our approach integrates multilingual OCR, large language models, and a novel region matching algorithm to localize answer spans at the block, line, word, and point levels. We introduce the Multi-Granular Visual Grounding (MGVG) benchmark, a curated test set of diverse circular notifications from various sectors, each manually annotated with fine-grained, human-verified labels across multiple granularities. Extensive experiments show that our method achieves state-of-the-art grounding accuracy, with line-level granularity providing the best balance between precision and recall. Ablation studies further highlight the benefits of multi-block and multi-line reasoning. Comparative evaluations reveal that leading vision-language models struggle with precise localization, underscoring the effectiveness of our structured, alignment-based approach. Our findings pave the way for more robust and interpretable document understanding systems in real-world, text-centric scenarios with multi-granular grounding support. Code and dataset are made available for future research.