2019/08/17 by Alexander Kuhnle, Kuhnle, Alexander, Ann Copestake +1
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Speech and dialogue systems #cs.AI #cs.CL #cs.CV #cs.LG
paper · pdf · doi:10.48550/arxiv.1908.06336
openalex publication_date 2019/08/17 · arxiv created 2019/10/22 · arxiv updated 2019/10/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Visual question answering (VQA) comprises a variety of language capabilities. The diagnostic benchmark dataset CLEVR has fueled progress by helping to better assess and distinguish models in basic abilities like counting, comparing and spatial reasoning in vitro. Following this approach, we focus on spatial language capabilities and investigate the question: what are the key ingredients to handle simple visual-spatial relations? We look at the SAN, RelNet, FiLM and MC models and evaluate their learning behavior on diagnostic data which is solely focused on spatial relations. Via comparative analysis and targeted model modification we identify what really is required to substantially improve upon the CNN-LSTM baseline.