Convolutional Neural Networks with Transformed Input based on Robust Tensor Network Decomposition

2018/11/20 by Jenn-Bing Ong, Wee-Keong Ng, Wee Keong Ng +5 · 2 citations
Computer Science · Mathematics · #Adversarial Robustness in Machine Learning #Adversarial system #Algorithm #Artificial intelligence #Big data #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Convolutional neural network #Data mining #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Mathematics #Parallel Computing and Optimization Techniques #Singular value decomposition #Tensor (intrinsic definition) #Tensor decomposition and applications #Theoretical computer science #cs.CV #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1812.02622

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2018/11/20 · arxiv created 2018/12/11 · openalex created_date 2018/12/11 · arxiv updated 2018/12/12 · openalex updated_date 2026/07/28

Abstract

Tensor network decomposition, originated from quantum physics to model entangled many-particle quantum systems, turns out to be a promising mathematical technique to efficiently represent and process big data in parsimonious manner. In this study, we show that tensor networks can systematically partition structured data, e.g. color images, for distributed storage and communication in privacy-preserving manner. Leveraging the sea of big data and metadata privacy, empirical results show that neighbouring subtensors with implicit information stored in tensor network formats cannot be identified for data reconstruction. This technique complements the existing encryption and randomization techniques which store explicit data representation at one place and highly susceptible to adversarial attacks such as side-channel attacks and de-anonymization. Furthermore, we propose a theory for adversarial examples that mislead convolutional neural networks to misclassification using subspace analysis based on singular value decomposition (SVD). The theory is extended to analyze higher-order tensors using tensor-train SVD (TT-SVD); it helps to explain the level of susceptibility of different datasets to adversarial attacks, the structural similarity of different adversarial attacks including global and localized attacks, and the efficacy of different adversarial defenses based on input transformation. An efficient and adaptive algorithm based on robust TT-SVD is then developed to detect strong and static adversarial attacks.

Citations

• Tensorizing Neural Networks
• Beating the Perils of Non-Convexity: Guaranteed Training of Neural Networks using Tensor Methods
• TT-cross approximation for multidimensional arrays
• Nonlinear total variation based noise removal algorithms
• Quantizing for minimum distortion
• ImageNet Large Scale Visual Recognition Challenge
• Least squares quantization in PCM
• Deep Multi-task Representation Learning: A Tensor Factorisation Approach
• Adversarial Examples in the Physical World
• Tensor Decomposition for Signal Processing and Machine Learning
• A study of the effect of JPG compression on adversarial images
• Efficient Processing of Deep Neural Networks: A Tutorial and Survey
• Spatially Transformed Adversarial Examples
• Robustness of Rotation-Equivariant Networks to Adversarial Perturbations
• Deep learning
• Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications
• Learning Multiple Layers of Features from Tiny Images

Related

• Privacy-preserving machine learning with tensor networks
• Protecting Big Data Privacy Using Randomized Tensor Network Decomposition and Dispersed Tensor Computation
• TensorShield: Tensor-based Defense Against Adversarial Attacks on Images
• Tensorization of neural networks for improved privacy and interpretability
• Model-Free Adversarial Purification via Coarse-To-Fine Tensor Network Representation