Exploring Potential Prompt Injection Attacks in Federated Military LLMs and Their Mitigation

2025/01/30 by Lee, Youngjoon, Park, Taehyun, Lee, Yunho +2 · 2 citations
#FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2501.18416

Abstract

Federated Learning (FL) is increasingly being adopted in military collaborations to develop Large Language Models (LLMs) while preserving data sovereignty. However, prompt injection attacks-malicious manipulations of input prompts-pose new threats that may undermine operational security, disrupt decision-making, and erode trust among allies. This perspective paper highlights four vulnerabilities in federated military LLMs: secret data leakage, free-rider exploitation, system disruption, and misinformation spread. To address these risks, we propose a human-AI collaborative framework with both technical and policy countermeasures. On the technical side, our framework uses red/blue team wargaming and quality assurance to detect and mitigate adversarial behaviors of shared LLM weights. On the policy side, it promotes joint AI-human policy development and verification of security protocols.

Cited by

• CourtGuard: A Local, Multiagent Prompt Injection Classifier
• LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systems

Related

• Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems
• Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition
• Federated Learning: Attacks, Defenses, Opportunities, and Challenges
• L2M-AID: Autonomous Cyber-Physical Defense by Fusing Semantic Reasoning of Large Language Models with Multi-Agent Reinforcement Learning (Preprint)
• Automating Prompt Leakage Attacks on Large Language Models Using Agentic Approach