2021/01/14 by Vinay Damodaran, Sharanya Chakravarthy, Damodaran, Vinay +14 · 1 citation
Computer Science · #Advanced Image and Video Retrieval Techniques #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #cs.CV #cs.LG
paper · pdf · doi:10.48550/arxiv.2101.05479
openalex publication_date 2021/01/14 · arxiv created 2021/01/17 · arxiv updated 2021/01/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Visual Question Answering (VQA) is of tremendous interest to the research community with important applications such as aiding visually impaired users and image-based search. In this work, we explore the use of scene graphs for solving the VQA task. We conduct experiments on the GQA dataset which presents a challenging set of questions requiring counting, compositionality and advanced reasoning capability, and provides scene graphs for a large number of images. We adopt image + question architectures for use with scene graphs, evaluate various scene graph generation techniques for unseen images, propose a training curriculum to leverage human-annotated and auto-generated scene graphs, and build late fusion architectures to learn from multiple image representations. We present a multi-faceted study into the use of scene graphs for VQA, making this work the first of its kind.