2017/12/18 by David J. Miller, Miller, David J., Yulia Wang +3
Biochemistry, Genetics and Molecular Biology · Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Bacillus and Francisella bacterial research #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning (cs.LG)
paper · pdf · doi:10.48550/arxiv.1712.06646
openalex publication_date 2017/12/18 · openalex created_date 2022/10/02 · openalex updated_date 2026/07/28
A significant threat to the recent, wide deployment of machine learning-based\nsystems, including deep neural networks (DNNs), is adversarial learning\nattacks. We analyze possible test-time evasion-attack mechanisms and show that,\nin some important cases, when the image has been attacked, correctly\nclassifying it has no utility: i) when the image to be attacked is (even\narbitrarily) selected from the attacker's cache; ii) when the sole recipient of\nthe classifier's decision is the attacker. Moreover, in some application\ndomains and scenarios it is highly actionable to detect the attack irrespective\nof correctly classifying in the face of it (with classification still performed\nif no attack is detected). We hypothesize that, even if human-imperceptible,\nadversarial perturbations are machine-detectable. We propose a purely\nunsupervised anomaly detector (AD) that, unlike previous works: i) models the\njoint density of a deep layer using highly suitable null hypothesis density\nmodels (matched in particular to the non- negative support for RELU layers);\nii) exploits multiple DNN layers; iii) leverages a "source" and "destination"\nclass concept, source class uncertainty, the class confusion matrix, and DNN\nweight information in constructing a novel decision statistic grounded in the\nKullback-Leibler divergence. Tested on MNIST and CIFAR-10 image databases under\nthree prominent attack strategies, our approach outperforms previous detection\nmethods, achieving strong ROC AUC detection accuracy on two attacks and better\naccuracy than recently reported for a variety of methods on the strongest (CW)\nattack. We also evaluate a fully white box attack on our system. Finally, we\nevaluate other important performance measures, such as classification accuracy,\nversus detection rate and attack strength.\n