On Swarm Leader Identification using Probing Policies

2025/12/20 by Stergios E. Bachoumas, Bachoumas, Stergios E., Panagiotis Artemiadis +1
Computer Science · Psychology · #Artificial Intelligence (cs.AI) #Distributed Control Multi-Agent Systems #FOS: Computer and information sciences #Multiagent Systems (cs.MA) #Reinforcement Learning in Robotics #Robotics (cs.RO) #Social Robot Interaction and HRI

paper · doi:10.48550/arxiv.2512.18146

openalex publication_date 2025/12/20 · openalex created_date 2025/12/24 · openalex updated_date 2026/07/28

Abstract

Identifying the leader within a robotic swarm is crucial, especially in adversarial contexts where leader concealment is necessary for mission success. This work introduces the interactive Swarm Leader Identification (iSLI) problem, a novel approach where an adversarial probing agent identifies a swarm's leader by physically interacting with its members. We formulate the iSLI problem as a Partially Observable Markov Decision Process (POMDP) and employ Deep Reinforcement Learning, specifically Proximal Policy Optimization (PPO), to train the prober's policy. The proposed approach utilizes a novel neural network architecture featuring a Timed Graph Relationformer (TGR) layer combined with a Simplified Structured State Space Sequence (S5) model. The TGR layer effectively processes graph-based observations of the swarm, capturing temporal dependencies and fusing relational information using a learned gating mechanism to generate informative representations for policy learning. Extensive simulations demonstrate that our TGR-based model outperforms baseline graph neural network architectures and exhibits significant zero-shot generalization capabilities across varying swarm sizes and speeds different from those used during training. The trained prober achieves high accuracy in identifying the leader, maintaining performance even in out-of-training distribution scenarios, and showing appropriate confidence levels in its predictions. Real-world experiments with physical robots further validate the approach, confirming successful sim-to-real transfer and robustness to dynamic changes, such as unexpected agent disconnections.

Citations

• Zero-Shot Reinforcement Learning Under Partial Observability
• Structured State Space Models for In-Context Reinforcement Learning
• Simplified State Space Layers for Sequence Modeling
• Model-based Multi-agent Reinforcement Learning: Recent Progress and Prospects
• Efficiently Modeling Long Sequences with Structured State Spaces
• Hiding Leader's Identity in Leader-Follower Navigation through\n Multi-Agent Reinforcement Learning
• Natural Emergence of Heterogeneous Strategies in Artificially Intelligent Competitive Teams
• Multi-Agent Actor-Critic with Hierarchical Graph Attention Network
• Time2Vec: Learning a Vector Representation of Time
• Learning Transferable Cooperative Behavior in Multi-Agent Teams
• Graph Attention Networks
• Proximal Policy Optimization Algorithms
• A simple neural network module for relational reasoning
• Deep Sets
• Layer Normalization
• Benchmarking Deep Reinforcement Learning for Continuous Control

Related

• Using Reinforcement Learning to Herd a Robotic Swarm to a Target Distribution
• Learning Robot Swarm Tactics over Complex Adversarial Environments
• Hiding Leader's Identity in Leader-Follower Navigation through\n Multi-Agent Reinforcement Learning
• Guided Deep Reinforcement Learning for Swarm Systems
• Learning of Coordination Policies for Robotic Swarms