2017/10/02 by Stephanie Zhou, Alane Suhr, Zhou, Stephanie +3
Computer Science · #Advanced Image and Video Retrieval Techniques #Computation and Language (cs.CL) #FOS: Computer and information sciences #Human Pose and Action Recognition #Multimodal Machine Learning Applications
paper · pdf · doi:10.48550/arxiv.1710.00453
openalex publication_date 2017/10/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Natural language provides a widely accessible and expressive interface for robotic agents. To understand language in complex environments, agents must reason about the full range of language inputs and their correspondence to the world. Such reasoning over language and vision is an open problem that is receiving increasing attention. While existing data sets focus on visual diversity, they do not display the full range of natural language expressions, such as counting, set reasoning, and comparisons. We propose a simple task for natural language visual reasoning, where images are paired with descriptive statements. The task is to predict if a statement is true for the given scene. This abstract describes our existing synthetic images corpus and our current work on collecting real vision data.