Fooling Vision and Language Models Despite Localization and Attention Mechanism

2017/09/25 by Xiaojun Xu, Xu, Xiaojun, Xinyun Chen +9
Computer Science · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.1709.08693

openalex publication_date 2017/09/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Adversarial attacks are known to succeed on classifiers, but it has been an open question whether more complex vision systems are vulnerable. In this paper, we study adversarial examples for vision and language models, which incorporate natural language understanding and complex structures such as attention, localization, and modular architectures. In particular, we investigate attacks on a dense captioning model and on two visual question answering (VQA) models. Our evaluation shows that we can generate adversarial examples with a high success rate (i.e., > 90%) for these models. Our work sheds new light on understanding adversarial attacks on vision systems which have a language component and shows that attention, bounding box localization, and compositional internal structures are vulnerable to adversarial attacks. These observations will inform future work towards building effective defenses.

Citations

• Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answering
• Gradient-based learning applied to document recognition
• Dynamic Memory Networks for Visual and Textual Question Answering
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding
• On the (Statistical) Detection of Adversarial Examples
• Tactics of Adversarial Attack on Deep Reinforcement Learning Agents
• MagNet: a Two-Pronged Defense against Adversarial Examples
• VQA: Visual Question Answering
• Adversarial Examples for Semantic Segmentation and Object Detection
• Practical Black-Box Attacks against Machine Learning
• Ask Your Neurons: A Neural-based Approach to Answering Questions about\n Images

Related

• Transferable Adversarial Attacks on Black-Box Vision-Language Models
• Revisiting CroPA: A Reproducibility Study and Enhancements for Cross-Prompt Adversarial Transferability in Vision-Language Models
• Seeing the Threat: Vulnerabilities in Vision-Language Models to Adversarial Attack
• Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks
• Break the Visual Perception: Adversarial Attacks Targeting Encoded Visual Tokens of Large Vision-Language Models