Jailbreaking Large Vision Language Models in Intelligent Transportation Systems
2025/11/17 by Das, Badhan Chandra, Jawad, Md Tasnim, Mia, Md Jueal +2
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences
paper · doi:10.48550/arxiv.2511.13892
Abstract
Large Vision Language Models (LVLMs) demonstrate strong capabilities in multimodal reasoning and many real-world applications, such as visual question answering. However, LVLMs are highly vulnerable to jailbreaking attacks. This paper systematically analyzes the vulnerabilities of LVLMs integrated in Intelligent Transportation Systems (ITS) under carefully crafted jailbreaking attacks. First, we carefully construct a dataset with harmful queries relevant to transportation, following OpenAI's prohibited categories to which the LVLMs should not respond. Second, we introduce a novel jailbreaking attack that exploits the vulnerabilities of LVLMs through image typography manipulation and multi-turn prompting. Third, we propose a multi-layered response filtering defense technique to prevent the model from generating inappropriate responses. We perform extensive experiments with the proposed attack and defense on the state-of-the-art LVLMs (both open-source and closed-source). To evaluate the attack method and defense technique, we use GPT-4's judgment to determine the toxicity score of the generated responses, as well as manual verification. Further, we compare our proposed jailbreaking method with existing jailbreaking techniques and highlight severe security risks involved with jailbreaking attacks with image typography manipulation and multi-turn prompting in the LVLMs integrated in ITS.
Citations
- CEQuest: Benchmarking Large Language Models for Construction Estimation
- Mapillary Vistas Validation for Fine-Grained Traffic Signs: A Benchmark Revealing Vision-Language Model Limitations
- System Prompt Extraction Attacks and Defenses in Large Language Models
- Large Language Models and Their Applications in Roadway Safety and Mobility Enhancement: A Comprehensive Review
- Qwen3 Technical Report
- Can Vision-Language Models Understand and Interpret Dynamic Gestures from Pedestrians? Pilot Datasets and Exploration Towards Instructive Nonverbal Commands for Cooperative Autonomous Vehicles
- Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmap
- Distributed LLMs and Multimodal Large Language Models: A Survey on Advances, Challenges, and Future Directions
- Qwen2.5-VL Technical Report
- Jailbreaking Multimodal Large Language Models via Shuffle Inconsistency
- Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directions
- MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notes
- Jailbreak Large Vision-Language Models Through Multi-Modal Linkage
- Visual Adversarial Attack on Vision-Language Models for Autonomous Driving
- MRJ-Agent: An Effective Jailbreak Agent for Multi-Round Dialogue
- CE-CoLLM: Efficient and Adaptive Large Language Models Through Cloud-Edge Collaboration
- GPT-4o System Card
- The Llama 3 Herd of Models
- LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models
- Chain of Attack: a Semantic-Driven Contextual Multi-Turn attacker for LLM
- Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack
- Security and Privacy Challenges of Large Language Models: A Survey
- MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models
- FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts
- Improved Baselines with Visual Instruction Tuning
- Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehicles
- NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models
- Vision-Language Models in Remote Sensing: Current Progress and Future Trends
- MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
- Visual Instruction Tuning
- Vision-Language Models for Vision Tasks: A Survey
- Vision-Language Models for Vision Tasks: A Survey
- GPT-4 Technical Report
- Learning Transferable Visual Models From Natural Language Supervision
Related