When Alignment Fails: Multimodal Adversarial Attacks on Vision-Language-Action Models

2025/11/20 by Yuping Yan, Y. G. Xie, Yan, Yuping +6 · 1 citation
Computer Science · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.2511.16203

openalex publication_date 2025/11/20 · openalex created_date 2025/11/23 · openalex updated_date 2026/07/28

Abstract

Vision-Language-Action models (VLAs) have recently demonstrated remarkable progress in embodied environments, enabling robots to perceive, reason, and act through unified multimodal understanding. Despite their impressive capabilities, the adversarial robustness of these systems remains largely unexplored, especially under realistic multimodal and black-box conditions. Existing studies mainly focus on single-modality perturbations and overlook the cross-modal misalignment that fundamentally affects embodied reasoning and decision-making. In this paper, we introduce VLA-Fool, a comprehensive study of multimodal adversarial robustness in embodied VLA models under both white-box and black-box settings. VLA-Fool unifies three levels of multimodal adversarial attacks: (1) textual perturbations through gradient-based and prompt-based manipulations, (2) visual perturbations via patch and noise distortions, and (3) cross-modal misalignment attacks that intentionally disrupt the semantic correspondence between perception and instruction. We further incorporate a VLA-aware semantic space into linguistic prompts, developing the first automatically crafted and semantically guided prompting framework. Experiments on the LIBERO benchmark using a fine-tuned OpenVLA model reveal that even minor multimodal perturbations can cause significant behavioral deviations, demonstrating the fragility of embodied multimodal alignment.

Citations

• Model-agnostic Adversarial Attack and Defense for Vision-Language-Action Models
• Pure Vision Language Action (VLA) Models: A Comprehensive Survey
• Eva-VLA: Evaluating Vision-Language-Action Models' Robustness Under Real-World Physical Variations
• Time-Unified Diffusion Policy with Action Discrimination for Robotic Manipulation
• Adversarial Attacks on Robotic Vision Language Action Models
• SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
• ForceVLA: Enhancing VLA Models with a Force-aware MoE for Contact-rich Manipulation
• VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning
• HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model
• SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model
• CLIPS: An Enhanced CLIP Framework for Learning with Synthetic Captions
• Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Robotics
• π₀: A Vision-Language-Action Flow Model for General Robot Control
• A Dual Process VLA: Efficient Robotic Manipulation Leveraging VLM
• RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation
• RoboNurse-VLA: Robotic Scrub Nurse System based on Vision-Language-Action Model
• Manipulation Facing Threats: Evaluating Physical Vulnerabilities in End-to-End Vision Language Action Models
• TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation
• OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Driving
• OpenVLA: An Open-Source Vision-Language-Action Model
• Octo: An Open-Source Generalist Robot Policy
• RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
• Universal and Transferable Adversarial Attacks on Aligned Language Models
• LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning
• Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
• Diffusion policy: Visuomotor policy learning via action diffusion
• RT-1: Robotics Transformer for Real-World Control at Scale
• BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• Utility-Optimized Local Differential Privacy Mechanisms for Distribution Estimation
• Explaining and Harnessing Adversarial Examples

Cited by

• VLSA: Vision-Language-Action Models with Plug-and-Play Safety Constraint Layer

Related

• Probing the Robustness of Vision-Language Pretrained Models: A Multimodal Adversarial Attack Approach
• Model-agnostic Adversarial Attack and Defense for Vision-Language-Action Models
• ADVEDM:Fine-grained Adversarial Attack against VLM-based Embodied Agents
• AttackVLA: Benchmarking Adversarial and Backdoor Attacks on Vision-Language-Action Models
• LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models