Evading Deep Learning-Based Malware Detectors via Obfuscation: A Deep Reinforcement Learning Approach

2024/02/04 by Etter, Brian, Hu, James Lee, Ebrahimi, Mohammedreza +3
#Artificial Intelligence (cs.AI) #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2402.02600

Abstract

Adversarial Malware Generation (AMG), the generation of adversarial malware variants to strengthen Deep Learning (DL)-based malware detectors has emerged as a crucial tool in the development of proactive cyberdefense. However, the majority of extant works offer subtle perturbations or additions to executable files and do not explore full-file obfuscation. In this study, we show that an open-source encryption tool coupled with a Reinforcement Learning (RL) framework can successfully obfuscate malware to evade state-of-the-art malware detection engines and outperform techniques that use advanced modification methods. Our results show that the proposed method improves the evasion rate from 27%-49% compared to widely-used state-of-the-art reinforcement learning-based methods.

Related

• Single-Shot Black-Box Adversarial Attacks Against Malware Detectors: A Causal Language Model Approach
• Adversarial Malware Binaries: Evading Deep Learning for Malware\n Detection in Executables
• Enhancing Robustness of Deep Neural Networks Against Adversarial Malware Samples: Principles, Framework, and AICS'2019 Challenge
• On the Effectiveness of Adversarial Samples against Ensemble Learning-based Windows PE Malware Detectors
• Harnessing the Speed and Accuracy of Machine Learning to Advance Cybersecurity