Demystifying RCE Vulnerabilities in LLM-Integrated Apps

2023/09/06 by Tong Liu, Zizhuang Deng, Guozhu Meng +2 · 5 voices · 9 citations
Decision Sciences · Computer Science · #Scientific Computing and Data Management #Security and Verification in Computing #Web Application Security Vulnerabilities

paper · pdf · doi:10.1145/3658644.3690338

Abstract

Large Language Models (LLMs) show promise in transforming software development, with a growing interest in integrating them into more intelligent apps. Frameworks like LangChain aid LLM-integrated app development, offering code execution utility/APIs for custom actions. However, these capabilities theoretically introduce Remote Code Execution (RCE) vulnerabilities, enabling remote code execution through prompt injections. No prior research systematically investigates these frameworks' RCE vulnerabilities or their impact on applications and exploitation consequences. Therefore, there is a huge research gap in this field.

Cited by

• "MCP Does Not Stand for Misuse Cryptography Protocol": Uncovering Cryptographic Misuse in Model Context Protocol at Scale
• Toward Understanding Security Issues in the Model Context Protocol Ecosystem
• Breaking Agent Backbones: Evaluating the Security of Backbone LLMs in AI Agents
• Cuckoo Attack: Stealthy and Persistent Attacks Against AI-IDE
• LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systems
• EchoLeak: The First Real-World Zero-Click Prompt Injection Exploit in a Production LLM System
• Securing Educational LLMs: A Generalised Taxonomy of Attacks on LLMs and DREAD Risk Assessment
• Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition
• Running in CIRCLE? A Simple Benchmark for LLM Code Interpreter Security

Discussions

• in absolutely the funniest outcome so far, you can send data to an LLM that pops a Remote Code Execution vulnerability [lemmy, 103 points, 14 comments]
• Tried the OCR thing I'm building on an academic paper: arxiv.org/abs/2309.029... I made a Quarto HTML from the resultant page markdown+HTML: rud.is/ex/academic.... [bsky, 6 points, 2 comments]
• Demystifying RCE Vulnerabilities in LLM-Integrated Apps [hn, 1 points, 0 comments]
• #RCE (remote-code-execution) vulnerabilities in #LLM (large-language-model) apps: arxiv.org/pdf/2309.02926 #infosec [bsky, 0 points, 1 comments]
• Demystifying RCE Vulnerabilities in LLM-Integrated Apps [bsky, 0 points, 0 comments]

Related

• From Vulnerabilities to Remediation: A Systematic Literature Review of LLMs in Code Security
• ACE: A Security Architecture for LLM-Integrated App Systems
• The Hidden Risks of LLM-Generated Web Application Code: A Security-Centric Evaluation of Code Generation Capabilities in Large Language Models
• Can We Trust Large Language Models Generated Code? A Framework for In-Context Learning, Security Patterns, and Code Evaluations Across Diverse LLMs
• Demystifying LLM Supply Chain Vulnerabilities in the Wild: Distribution, Root Cause, and Real-World Impact