Detecting Statistical Interactions from Neural Network Weights

2017/05/14 by Michael Tsang, Tsang, Michael, Dehua Cheng +3 · 9 citations
Computer Science · #Anomaly Detection Techniques and Applications #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.1705.04977

openalex publication_date 2017/05/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Interpreting neural networks is a crucial and challenging task in machine learning. In this paper, we develop a novel framework for detecting statistical interactions captured by a feedforward multilayer neural network by directly interpreting its learned weights. Depending on the desired interactions, our method can achieve significantly better or similar interaction detection performance compared to the state-of-the-art without searching an exponential solution space of possible interactions. We obtain this accuracy and efficiency by observing that interactions between input features are created by the non-additive effect of nonlinear activation functions, and that interacting paths are encoded in weight matrices. We demonstrate the performance of our method and the importance of discovered interactions via experimental results on both synthetic datasets and real-world application datasets.

Citations

• Sparse spatial autoregressions
• Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups
• European Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
• Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
• RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism
• The Mythos of Model Interpretability

Cited by

• A Model of Causal Explanation on Neural Networks for Tabular Data
• Order-Level Attention Similarity Across Language Models: A Latent Commonality
• Bayesian Neural Networks for Functional ANOVA model
• Guiding Mixture-of-Experts with Temporal Multimodal Interactions
• Why is topology hard to learn?
• CUBE: Contrastive Understanding by Balanced Experiments
• Tree-like Pairwise Interaction Networks
• High Dimensional Model Explanations: an Axiomatic Approach
• Periodic Spectral Ergodicity: A Complexity Measure for Deep Neural\n Networks and Neural Architecture Search

Related

• Sparse Interaction Additive Networks via Feature Interaction Detection and Sparse Selection
• Towards Interaction Detection Using Topological Analysis on Neural Networks
• Tree-like Pairwise Interaction Networks
• High-dimensional Interactions Detection with Sparse Principal Hessian Matrix
• Interpreting Multivariate Shapley Interactions in DNNs