Escalation Risks from Language Models in Military and Diplomatic Decision-Making

2024/01/07 by Juan-Pablo Rivera, Gabriel Mukobi, Anka Reuel +3 · 20 voices · 14 citations
Economics, Econometrics and Finance · Social Sciences · #Economic Sanctions and International Relations #Terrorism, Counterterrorism, and Political Violence #International Relations and Foreign Policy

paper · pdf · doi:10.1145/3630106.3658942

Abstract

Governments are increasingly considering integrating autonomous AI agents in high-stakes military and foreign-policy decision-making, especially with the emergence of advanced generative AI models like GPT-4. Our work aims to scrutinize the behavior of multiple AI agents in simulated wargames, specifically focusing on their predilection to take escalatory actions that may exacerbate multilateral conflicts. Drawing on political science and international relations literature about escalation dynamics, we design a novel wargame simulation and scoring framework to assess the escalation risks of actions taken by these agents in different scenarios. Contrary to prior studies, our research provides both qualitative and quantitative insights and focuses on large language models (LLMs). We find that all five studied off-the-shelf LLMs show forms of escalation and difficult-to-predict escalation patterns. We observe that models tend to develop arms-race dynamics, leading to greater conflict, and in rare cases, even to the deployment of nuclear weapons. Qualitatively, we also collect the models’ reported reasoning for chosen actions and observe worrying justifications based on deterrence and first-strike tactics. Given the high stakes of military and foreign-policy contexts, we recommend further examination and cautious consideration before deploying autonomous language model agents for strategic military or diplomatic decision-making.

Cited by

• AI Arms and Influence: Frontier Models Exhibit Sophisticated Reasoning in Simulated Nuclear Crises
• Misinformation by Omission: The Need for More Environmental Transparency in AI
• When Two LLMs Debate, Both Think They'll Win
• The Weaponization of Computer Vision: Tracing Military-Surveillance Ties through Conference Sponsorship
• Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordia
• Can LLMs Threaten Human Survival? Benchmarking Potential Existential Threats from LLMs via Prefix Completion
• A Criminology of Machines
• Artificial Intelligence and U.S. Military Cadets’ Attitudes About Future War
• ToxScreen: Detecting Whether an LLM Has Been Poisoned
• The Role of Social Learning and Collective Norm Formation in Fostering Cooperation in LLM Multi-Agent Systems
• Governing Automated Strategic Intelligence
• Mitigation of Gender and Ethnicity Bias in AI-Generated Stories through Model Explanations
• Managing Escalation in Off-the-Shelf Large Language Models
• Against racing to AGI: Cooperation, deterrence, and catastrophic risks

Discussions

• What happens if you put an AI in charge of national defense? In war games, LLMs tend to escalate & do arms races. Base models are more aggressive & unpredictable. The authors speculate that it is bec [bsky, 124 points, 8 comments]
• Not what you want to hear: Study finds that AI systems used for military decision making are likely to escalate. “We observe that models tend to develop arms-race dynamics, leading to greater conflic [bsky, 102 points, 14 comments]
• Escalation Risks from Language Models in Military and Diplomatic Decision-Making [hn, 52 points, 12 comments]
• Governments are increasingly using AI in military decision-making: study finds AI war games escalate conflicts, push arms races, and justify nuclear strikes. “We find that all five studied off-the-she [bsky, 7 points, 1 comments]
• Seems like the actual existential risk associated with our current batch of models is less that they are actually intelligent and more that we will trick ourselves into thinking they are. arxiv.org/ab [bsky, 6 points, 0 comments]
• Escalation Risks from Language Models in Military and Diplomatic Decision-Making [hn, 2 points, 0 comments]
• AI's dark side? Interesting study finds #LLMs in wargames can initiate dangerous escalation scenarios. Work needed re: the future of #AI in military conflict prediction and prevention arxiv.org/abs/24 [bsky, 1 points, 0 comments]
• Das ist so sinnvoll wie unrealistisch. Ich hatte mich (nur) auf die im Artikel erwähnte Studie (arxiv.org/pdf/2401.03408) bezogen: „Von der Stange“-LLMs spielen Konflikt-Simulationen. Das hat nur weni [bsky, 1 points, 0 comments]
• A better link: arxiv.org/abs/2401.03408 [bsky, 1 points, 0 comments]
• "We find that all five studied off-the-shelf [military-related] LLMs show forms of escalation and difficult-to-predict escalation patterns.. models tend to develop arms-race dynamics, leading to great [bsky, 1 points, 0 comments]
• Lendo com até bastante atenção um estudo do ano passado meio em tom lúdico do uso de IA para planejamento e tomada de decisões em guerra arxiv.org/pdf/2401.03408 [bsky, 1 points, 0 comments]
• Will AI lead us to war? Check out this paper with a superstar team at Stanford that compares LLMs when placed in a crisis scenario. Which goes nuclear? Which appeases? What does that mean for LLMs [bsky, 0 points, 0 comments]
• arxiv.org/abs/2401.03408 [bsky, 0 points, 0 comments]
• A new study designed “a novel wargame simulation and scoring framework to assess the escalation risks of actions taken by [AIs] in different scenarios”. Across all 5 tested models, “LLMs show forms of [bsky, 0 points, 0 comments]
• Abstract page: arxiv.org/abs/2401.03408 [bsky, 0 points, 0 comments]
• "Escalation Risks from Language Models in Military & Diplomatic Decision-Making" - Not sure anyone is giving language models authority to make military or diplomatic decisions (in the study it even us [bsky, 0 points, 0 comments]
• Ik had nu pas even de tijd om dit interessante onderzoek te bekijken: arxiv.org/pdf/2401.034... Chatbots blijken niet geschikt voor de-escalatie van nucleair conflict (joh) [bsky, 0 points, 0 comments]
• arxiv.org/abs/2401.03408 [bsky, 0 points, 0 comments]
• Escalation Risks from Language Models in Military and Diplomatic Decision-Making arxiv.org/pdf/2401.03408 [bsky, 0 points, 0 comments]
• I still find it wild that you can do so much interesting research on trying to understand one model but it is important as soon they are going to get us into war https://arxiv.org/abs/2401.03408 [bsky, 0 points, 0 comments]

Related

• AI Arms and Influence: Frontier Models Exhibit Sophisticated Reasoning in Simulated Nuclear Crises
• Impact of Military Artificial Intelligence on Nuclear Escalation Risk
• Shall We Play a Game? Language Models for Open-ended Wargames
• Human vs. Machine: Behavioral Differences Between Expert Humans and Language Models in Wargame Simulations
• Managing Escalation in Off-the-Shelf Large Language Models