Where To Look: Focus Regions for Visual Question Answering

2015/11/23 by Shih, Kevin J., Singh, Saurabh, Hoiem, Derek · 3 citations
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.1511.07394

Abstract

We present a method that learns to answer visual questions by selecting image regions relevant to the text-based query. Our method exhibits significant improvements in answering questions such as "what color," where it is necessary to evaluate a specific location, and "what room," where it selectively identifies informative image regions. Our model is tested on the VQA dataset which is the largest human-annotated visual question answering dataset to our knowledge.

Cited by

• DepthFocus: Controllable Depth Estimation for See-Through Scenes
• Vision Generalist Model: A Survey
• Light as Deception: GPT-driven Natural Relighting Against Vision-Language Pre-training Models

Related

• Acknowledging Focus Ambiguity in Visual Questions
• A Focused Dynamic Attention Model for Visual Question Answering
• Visual Question Answering Using Semantic Information from Image\n Descriptions
• Answer Questions with Right Image Regions: A Visual Attention Regularization Approach
• FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question Answering