Depending on yourself when you should: Mentoring LLM with RL agents to become the master in cybersecurity games

2024/03/26 by Yikuan Yan, Yan, Yikuan, Yaolun Zhang +3 · 8 citations
Computer Science · Social Sciences · #Access Control and Trust #Artificial Intelligence (cs.AI) #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Information and Cyber Security #Multi-Agent Systems and Negotiation #Multiagent Systems (cs.MA)

paper · pdf · doi:10.48550/arxiv.2403.17674

openalex publication_date 2024/03/26 · openalex created_date 2024/03/28 · openalex updated_date 2026/07/28

Abstract

Integrating LLM and reinforcement learning (RL) agent effectively to achieve complementary performance is critical in high stake tasks like cybersecurity operations. In this study, we introduce SecurityBot, a LLM agent mentored by pre-trained RL agents, to support cybersecurity operations. In particularly, the LLM agent is supported with a profile module to generated behavior guidelines, a memory module to accumulate local experiences, a reflection module to re-evaluate choices, and an action module to reduce action space. Additionally, it adopts the collaboration mechanism to take suggestions from pre-trained RL agents, including a cursor for dynamic suggestion taken, an aggregator for multiple mentors' suggestions ranking and a caller for proactive suggestion asking. Building on the CybORG experiment framework, our experiences show that SecurityBot demonstrates significant performance improvement compared with LLM or RL standalone, achieving the complementary performance in the cybersecurity games.

Cited by

• Evaluating the Quality of Randomness and Entropy in Tasks Supported by Large Language Models
• Incident Response Planning Using a Lightweight Large Language Model with Reduced Hallucination
• MetaAgent: Automatically Constructing Multi-Agent Systems Based on Finite State Machines
• Adaptive Network Security Policies via Belief Aggregation and Rollout
• AnyMAC: Cascading Flexible Multi-Agent Collaboration via Next-Agent Prediction
• Large Language Models are Autonomous Cyber Defenders
• Agentic Incident Response through Digital Twin-Enhanced Multiscale Planning
• Internet of Agents: Fundamentals, Applications, and Challenges

Related

• Large Language Model Integration with Reinforcement Learning to Augment Decision-Making in Autonomous Cyber Operations
• Guidelines for Applying RL and MARL in Cybersecurity Applications
• A Multiagent CyberBattleSim for RL Cyber Operation Agents
• CyGIL: A Cyber Gym for Training Autonomous Agents over Emulated Network Systems
• Network Environment Design for Autonomous Cyberdefense