2024/09/20 by Leblanc, Samuel, Aiky Rasolomanana, Marco Armenta +2
Computer Science · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE) #Representation Theory (math.RT)
paper · pdf · doi:10.48550/arxiv.2409.13163
openalex publication_date 2024/09/20 · openalex created_date 2024/10/26 · openalex updated_date 2026/07/28
We introduce a novel mathematical framework for analyzing neural networks using tools from quiver representation theory. This framework enables us to quantify the similarity between a new data sample and the training data, as perceived by the neural network. By leveraging the induced quiver representation of a data sample, we capture more information than traditional hidden layer outputs. This quiver representation abstracts away the complexity of the computations of the forward pass into a single matrix, allowing us to employ simple geometric and statistical arguments in a matrix space to study neural network predictions. Our mathematical results are architecture-agnostic and task-agnostic, making them broadly applicable. As proof of concept experiments, we apply our results for the MNIST and FashionMNIST datasets on the problem of detecting adversarial examples on different MLP architectures and several adversarial attack methods. Our experiments can be reproduced with our \hrefhttps://github.com/MarcoArmenta/Hidden-Activations-are-not-Enoughpublicly available repository.