Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs

2023/07/19 by Eugene Bagdasaryan, Tsung-Yin Hsieh, Bagdasaryan, Eugene +5 · 15 citations
Computer Science · Engineering · #Security and Verification in Computing #Digital and Cyber Forensics #Electrostatic Discharge in Electronics

paper · pdf · doi:10.48550/arxiv.2307.10490

Abstract

We demonstrate how images and sounds can be used for indirect prompt and instruction injection in multi-modal LLMs. An attacker generates an adversarial perturbation corresponding to the prompt and blends it into an image or audio recording. When the user asks the (unmodified, benign) model about the perturbed image or audio, the perturbation steers the model to output the attacker-chosen text and/or make the subsequent dialog follow the attacker's instruction. We illustrate this attack with several proof-of-concept examples targeting LLaVa and PandaGPT.

Cited by

• Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoring
• Attention is All You Need to Defend Against Indirect Prompt Injection Attacks in LLMs
• Toward a Safe Internet of Agents
• Reimagining Safety Alignment with An Image
• Prevalence of Security and Privacy Risk-Inducing Usage of AI-based Conversational Agents
• The Promptware Kill Chain: How Prompt Injections Gradually Evolved Into a Multistep Malware Delivery Mechanism
• Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges
• Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed
• Piggybacking on Perception: Stealthy Concurrent Audio Prompt Injections against Multimodal LLM Agents
• Image-based Prompt Injection: Hijacking Multimodal LLMs through Visually Embedded Adversarial Instructions
• Defense-to-Attack: Bypassing Weak Defenses Enables Stronger Jailbreaks in Vision-Language Models
• Invitation Is All You Need! Promptware Attacks Against LLM-Powered Assistants in Production Are Practical and Dangerous
• Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation
• Special-Character Adversarial Attacks on Open-Source Language Model
• Simulated Ensemble Attack: Transferring Jailbreaks Across Fine-tuned Vision-Language Models

Related

• Image-based Prompt Injection: Hijacking Multimodal LLMs through Visually Embedded Adversarial Instructions
• Con Instruction: Universal Jailbreaking of Multimodal Large Language Models via Non-Textual Modalities
• Adversarial Illusions in Multi-Modal Embeddings
• FMM-Attack: A Flow-based Multi-modal Adversarial Attack on Video-based LLMs
• ARGUS: Defending Against Multimodal Indirect Prompt Injection via Steering Instruction-Following Behavior