Policy-Value Guided MDP-MCTS Framework for Cyber Kill-Chain Inference

2025/12/17 by Singh, Chitraksh, Dhanraj, Monisha, Huang, Ken
Computer Science · #Advanced Malware Detection Techniques #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Information and Cyber Security #Network Security and Intrusion Detection

paper · doi:10.48550/arxiv.2512.15150

openalex publication_date 2025/12/17 · openalex created_date 2025/12/19 · openalex updated_date 2026/07/28

Abstract

Threat analysts routinely rely on natural-language reports that describe attacker actions without enumerating the full kill chain or the dependencies between phases, making automated reconstruction of ATT&CK consistent intrusion paths a difficult open problem. We propose a reasoning framework that infers complete seven-phase kill chains by coupling phase-conditioned semantic priors from Transformer models with a symbolic Markov Decision Process and an AlphaZero-style Monte Carlo Tree Search guided by a Policy-Value Network. The framework enforces semantic relevance, phase cohesion, and transition plausibility through a multi-objective reward function while allowing search to explore alternative interpretations of the CTI narrative. Applied to three real intrusions FIN6, APT24, and UNC1549 the approach yields kill chains that surpass Transformer baselines in semantic fidelity and operational coherence, and frequently align with expert-selected TTPs. Our results demonstrate that combining contextual embeddings with search-based decision-making offers a practical path toward automated, interpretable kill-chain reconstruction for cyber defense.

Citations

• KillChainGraph: ML Framework for Predicting and Mapping ATT&CK Techniques
• Planning Stealthy Backdoor Attacks in MDPs with Observation-Based Triggers
• On Almost-Sure Intention Deception Planning that Exploits Imperfect Observers
• Markov Decision Process For Automatic Cyber Defense
• A Markov Game Model for AI-based Cyber Security Attack Mitigation
• Optimal Attack against Cyber-Physical Control Systems with Reactive Attack Mitigation

Related

• KillChainGraph: ML Framework for Predicting and Mapping ATT&CK Techniques
• An Automated Framework for Extracting Reachable Attack Chains from Cyber Threat Intelligence Reports
• CRUcialG: Reconstruct Integrated Attack Scenario Graphs by Cyber Threat Intelligence Reports
• Principled Data-Driven Decision Support for Cyber-Forensic Investigations
• The Procedural Semantics Gap in Structured CTI: A Measurement-Driven STIX Analysis for APT Emulation