Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning

2017/12/31 by Battista Biggio, Fabio Roli · 3 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · #Advanced Malware Detection Techniques #Adversarial Robustness in Machine Learning #Adversarial machine learning #Adversarial system #Artificial intelligence #Artificial neural network #Bacillus and Francisella bacterial research #Computer science #Computer security #Deep learning #Facial recognition system #Feature extraction #Field (mathematics) #Machine learning #Malware #cs.CR #cs.CV #cs.GT #cs.LG

paper · pdf · doi:10.1016/j.patcog.2018.07.023

Accepted for publication on Pattern Recognition, 2018

arxiv created 2018/07/19 · openalex publication_date 2018/07/21 · crossref created 2018/07/21 · arxiv updated 2018/07/24 · crossref issued 2018/12/01 · crossref published 2018/12/01 · crossref published-print 2018/12/01 · crossref deposited 2020/01/11 · openalex created_date 2025/10/10 · crossref indexed 2026/08/04 · openalex updated_date 2026/08/05

Abstract

Learning-based pattern classifiers, including deep networks, have shown impressive performance in several application domains, ranging from computer vision to cybersecurity. However, it has also been shown that adversarial input perturbations carefully crafted either at training or at test time can easily subvert their predictions. The vulnerability of machine learning to such wild patterns (also referred to as adversarial examples), along with the design of suitable countermeasures, have been investigated in the research field of adversarial machine learning. In this work, we provide a thorough overview of the evolution of this research area over the last ten years and beyond, starting from pioneering, earlier work on the security of non-deep learning algorithms up to more recent work aimed to understand the security properties of deep learning algorithms, in the context of computer vision and cybersecurity tasks. We report interesting connections between these apparently-different lines of work, highlighting common misconceptions related to the security evaluation of machine-learning algorithms. We review the main threat models and attacks defined to this end, and discuss the main limitations of current work, along with the corresponding future challenges towards the design of more secure learning algorithms.

Citations

• A unifying view on dataset shift in classification
• The security of machine learning
• <title>Impact of artificial "gummy" fingers on fingerprint systems</title>
• Security Analysis of Online Centroid Anomaly Detection
• A Unified Gradient Regularization Family for Adversarial Examples
• The Mythos of Model Interpretability
• Safety Verification of Deep Neural Networks
• Understanding Black-box Predictions via Influence Functions
• Certified Defenses for Data Poisoning Attacks
• MagNet: a Two-Pronged Defense against Adversarial Examples
• Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning
• Recent advances in convolutional neural networks
• Provable defenses against adversarial examples via the convex outer adversarial polytope

Cited by

• Adversarial observations in probabilistic State-Space Models for robust Reinforcement Learning
• Game Theoretical Adversarial Deep Learning With Variational Adversaries
• Square Attack: A Query-Efficient Black-Box Adversarial Attack via Random Search
• Adversarial machine learning [wikipedia]