Scientific Calculator for Designing Trojan Detectors in Neural Networks

2020/06/05 by Peter Bajcsy, Bajcsy, Peter, Nicholas J. Schaub +3
Computer Science · #Adversarial Robustness in Machine Learning #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #Physical Unclonable Functions (PUFs) and Hardware Security

paper · pdf · doi:10.48550/arxiv.2006.03707

openalex publication_date 2020/06/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This work presents a web-based interactive neural network (NN) calculator and a NN inefficiency measurement that has been investigated for the purpose of detecting trojans embedded in NN models. This NN Calculator is designed on top of TensorFlow Playground with in-memory storage of data and NN graphs plus coefficients. It is "like a scientific calculator" with analytical, visualization, and output operations performed on training datasets and NN architectures. The prototype is aaccessible at https://pages.nist.gov/nn-calculator. The analytical capabilities include a novel measurement of NN inefficiency using modified Kullback-Liebler (KL) divergence applied to histograms of NN model states, as well as a quantification of the sensitivity to variables related to data and NNs. Both NN Calculator and KL divergence are used to devise a trojan detector approach for a variety of trojan embeddings. Experimental results document desirable properties of the KL divergence measurement with respect to NN architectures and dataset perturbations, as well as inferences about embedded trojans.

Citations

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Learning with Limited Numerical Precision
• Representation Learning: A Review and New Perspectives
• Understanding deep convolutional networks
• Binarized Neural Networks
• Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures
• Direct-Manipulation Visualization of Deep Networks
• What Does Explainable AI Really Mean? A New Conceptualization of Perspectives
• Mathematics of Deep Learning
• Sensitivity and Generalization in Neural Networks: an Empirical Study
• Learning Deep Features for Discriminative Localization
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural\n Networks
• Visualizing and Understanding Convolutional Networks

Related

• Baseline Pruning-Based Approach to Trojan Detection in Neural Networks
• Topological Detection of Trojaned Neural Networks
• TAD: Trigger Approximation based Black-box Trojan Detection for AI
• A Survey of Trojan Attacks and Defenses to Deep Neural Networks
• Cassandra: Detecting Trojaned Networks from Adversarial Perturbations