The security of machine learning

2010/05/19 by Marco Barreno, Blaine Nelson, Anthony D. Joseph +1 · 13 citations
Computer Science · #Network Security and Intrusion Detection #Advanced Malware Detection Techniques #Spam and Phishing Detection #Computer science #Adaptability #Exploit #Taxonomy (biology) #Adversarial machine learning #Machine learning #Vulnerability (computing) #Artificial intelligence #Computer security #Deep learning

paper · pdf · doi:10.1007/s10994-010-5188-5

openalex publication_date 2010/05/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Machine learning’s ability to rapidly evolve to changing and complex situations has helped it become a fundamental tool for computer security. That adaptability is also a vulnerability: attackers can exploit machine learning systems. We present a taxonomy identifying and analyzing attacks against machine learning systems. We show how these classes influence the costs for the attacker and defender, and we give a formal structure defining their interaction. We use our framework to survey and analyze the literature of attacks against machine learning systems. We also illustrate our taxonomy by showing how it can guide attacks against SpamBayes, a popular statistical spam filter. Finally, we discuss how our taxonomy suggests new lines of defenses.

Citations

• A Mathematical Theory of Communication
• Questions and Answers
• A theory of the learnable

Cited by

• Audio Adversarial Examples: Targeted Attacks on Speech-to-Text
• Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
• Machine learning in adversarial environments
• Security Evaluation of Support Vector Machines in Adversarial Environments
• Timing Attacks on Machine Learning: State of the Art
• High-Dimensional Robust Mean Estimation via Gradient Descent
• Applications of machine learning in cryptography
• Recent advances in feature selection and its applications
• Security Evaluation of Pattern Classifiers under Attack
• Certified Defenses against Adversarial Examples
• Towards a robust and trustworthy machine learning system development: An engineering perspective
• Evaluating and Improving Adversarial Robustness of Machine Learning-Based Network Intrusion Detectors
• Adversarial machine learning [wikipedia]

Related

• Attacking Machine Learning models as part of a cyber kill chain
• Threats, Vulnerabilities, and Controls of Machine Learning Based Systems: A Survey and Taxonomy
• A Survey on Resilient Machine Learning
• Towards the Science of Security and Privacy in Machine Learning
• Defenses in Adversarial Machine Learning: A Survey