AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attacks

2024/03/02 by Jiacen Xu, Jack W. Stokes, Xu, Jiacen +13 · 20 citations
Computer Science · #Artificial Intelligence (cs.AI) #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Network Security and Intrusion Detection

paper · pdf · doi:10.48550/arxiv.2403.01038

openalex publication_date 2024/03/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Large language models (LLMs) have demonstrated impressive results on natural language tasks, and security researchers are beginning to employ them in both offensive and defensive systems. In cyber-security, there have been multiple research efforts that utilize LLMs focusing on the pre-breach stage of attacks like phishing and malware generation. However, so far there lacks a comprehensive study regarding whether LLM-based systems can be leveraged to simulate the post-breach stage of attacks that are typically human-operated, or "hands-on-keyboard" attacks, under various attack techniques and environments. As LLMs inevitably advance, they may be able to automate both the pre- and post-breach attack stages. This shift may transform organizational attacks from rare, expert-led events to frequent, automated operations requiring no expertise and executed at automation speed and scale. This risks fundamentally changing global computer security and correspondingly causing substantial economic impacts, and a goal of this work is to better understand these risks now so we can better prepare for these inevitable ever-more-capable LLMs on the horizon. On the immediate impact side, this research serves three purposes. First, an automated LLM-based, post-breach exploitation framework can help analysts quickly test and continually improve their organization's network security posture against previously unseen attacks. Second, an LLM-based penetration test system can extend the effectiveness of red teams with a limited number of human analysts. Finally, this research can help defensive systems and teams learn to detect novel attack behaviors preemptively before their use in the wild....

Cited by

• From Rookie to Expert: Manipulating LLMs for Automated Vulnerability Exploitation in Enterprise Software
• Bounty Hunter: Autonomous, Comprehensive Emulation of Multi-Faceted Adversaries
• PentestEval: Benchmarking LLM-based Penetration Testing with Modular and Stage-Level Design
• Automated Penetration Testing with LLM Agents and Classical Planning
• Hiding in the AI Traffic: Abusing MCP for LLM-Powered Agentic Red Teaming
• N-GLARE: An Non-Generative Latent Representation-Efficient LLM Safety Evaluator
• PoCo: Agentic Proof-of-Concept Exploit Generation for Smart Contracts
• A Comprehensive Evaluation and Practice of System Penetration Testing
• Network Intrusion Detection: Evolution from Conventional Approaches to LLM Collaboration and Emerging Risks
• PACEbench: A Framework for Evaluating Practical AI Cyber-Exploitation Capabilities
• Living Off the LLM: How LLMs Will Change Adversary Tactics
• Red-Teaming the Agentic Red-Team
• AutoPentester: An LLM Agent-based Framework for Automated Pentesting
• From Capabilities to Performance: Evaluating Key Functional Properties of LLM Architectures in Penetration Testing
• xOffense: An AI-driven autonomous penetration testing framework with offensive knowledge-enhanced LLMs and multi agent systems
• Beyond Data Privacy: New Privacy Risks for Large Language Models
• Shell or Nothing: Real-World Benchmarks and Memory-Activated Agents for Automated Penetration Testing
• Neuro-Symbolic AI for Cybersecurity: State of the Art, Challenges, and Opportunities
• CyGATE: Game-Theoretic Cyber Attack-Defense Engine for Patch Strategy Optimization
• Can We End the Cat-and-Mouse Game? Simulating Self-Evolving Phishing Attacks with LLMs and Genetic Algorithms

Related

• Large Language Models for Cyber Security: A Systematic Literature Review
• Forewarned is Forearmed: A Survey on Large Language Model-based Agents in Autonomous Cyberattacks
• Large Language Models for Cyber Security
• Hacking Back the AI-Hacker: Prompt Injection as a Defense Against LLM-driven Cyberattacks
• From Text to MITRE Techniques: Exploring the Malicious Use of Large Language Models for Generating Cyber Attack Payloads