Transformer-based Spatial Grounding: A Comprehensive Survey

2025/07/17 by Ijazul Haq, Muhammad Saqib, Haq, Ijazul +3
Computer Science · Engineering · #Advanced Computational Techniques and Applications #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Power Systems and Technologies

paper · pdf · doi:10.48550/arxiv.2507.12739

openalex publication_date 2025/07/17 · openalex created_date 2025/10/18 · openalex updated_date 2026/07/28

Abstract

Spatial grounding, the process of associating natural language expressions with corresponding image regions, has rapidly advanced due to the introduction of transformer-based models, significantly enhancing multimodal representation and cross-modal alignment. Despite this progress, the field lacks a comprehensive synthesis of current methodologies, dataset usage, evaluation metrics, and industrial applicability. This paper presents a systematic literature review of transformer-based spatial grounding approaches from 2018 to 2025. Our analysis identifies dominant model architectures, prevalent datasets, and widely adopted evaluation metrics, alongside highlighting key methodological trends and best practices. This study provides essential insights and structured guidance for researchers and practitioners, facilitating the development of robust, reliable, and industry-ready transformer-based spatial grounding models.

Citations

• Towards Visual Grounding: A Survey
• Q-Ground: Image Quality Grounding with Large Multi-modality Models
• GLaMM: Pixel Grounding Large Multimodal Model
• Detecting and Grounding Multi-Modal Media Manipulation
• Annotated Point Clouds and Images from a Spatial-Semantic Perception Pipeline: Robotized Deconstruction at the Reference Construction Site, Aachen
• Grounding Language Models to Images for Multimodal Inputs and Outputs
• GLIGEN: Open-Set Grounded Text-to-Image Generation
• SiRi: A Simple Selective Retraining Mechanism for Transformer-based Visual Grounding
• TransVG++: End-to-End Visual Grounding with Language Conditioned Vision Transformer
• Collaborative Transformers for Grounded Situation Recognition
• Bottom Up Top Down Detection Transformers for Language Grounding in Images and Point Clouds
• Multimodal Incremental Transformer with Visual Grounding for Visual Dialogue Generation
• Word2Pix: Word to Pixel Cross Attention Transformer in Visual Grounding
• Referring Transformer: A One-step Approach to Multi-task Visual Grounding
• TransVG: End-to-End Visual Grounding with Transformers
• TIGEr: Text-to-Image Grounding for Image Caption Evaluation
• VL-BERT: Pre-training of Generic Visual-Linguistic Representations
• ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
• Unsupervised Textual Grounding: Linking Words to Image Concepts
• Systematic literature reviews in software engineering – A systematic literature review

Related

• TransVG: End-to-End Visual Grounding with Transformers
• YORO -- Lightweight End to End Visual Grounding
• TransVG++: End-to-End Visual Grounding with Language Conditioned Vision Transformer
• Visual Grounding with Transformers
• GroundingGPT:Language Enhanced Multi-modal Grounding Model