Shift-Invariant Attribute Scoring for Kolmogorov-Arnold Networks via Shapley Value

2025/10/02 by Fan, Wangxuan, Wang, Ching, Li, Siqi +1 · 1 citation
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2510.01663

Abstract

For many real-world applications, understanding feature-outcome relationships is as crucial as achieving high predictive accuracy. While traditional neural networks excel at prediction, their black-box nature obscures underlying functional relationships. Kolmogorov--Arnold Networks (KANs) address this by employing learnable spline-based activation functions on edges, enabling recovery of symbolic representations while maintaining competitive performance. However, KAN's architecture presents unique challenges for network pruning. Conventional magnitude-based methods become unreliable due to sensitivity to input coordinate shifts. We propose ShapKAN, a pruning framework using Shapley value attribution to assess node importance in a shift-invariant manner. Unlike magnitude-based approaches, ShapKAN quantifies each node's actual contribution, ensuring consistent importance rankings regardless of input parameterization. Extensive experiments on synthetic and real-world datasets demonstrate that ShapKAN preserves true node importance while enabling effective network compression. Our approach improves KAN's interpretability advantages, facilitating deployment in resource-constrained environments.

Citations

• shapiq: Shapley Interactions for Machine Learning
• A preliminary study on continual learning in computer vision using Kolmogorov-Arnold Networks
• Efficient prediction of potential energy surface and physical properties with Kolmogorov-Arnold Networks
• Application of Kolmogorov-Arnold Networks in high energy physics
• KAN 2.0: Kolmogorov-Arnold Networks Meet Science
• KAN or MLP: A Fairer Comparison
• Kolmogorov-Arnold Networks for Time Series: Bridging Predictive Power and Interpretability
• Kolmogorov-Arnold Networks are Radial Basis Function Networks
• KAN: Kolmogorov-Arnold Networks
• A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations
• A Survey on Deep Neural Network Pruning: Taxonomy, Comparison, Analysis, and Recommendations
• Structured Pruning for Deep Convolutional Neural Networks: A Survey
• The Shapley Value in Machine Learning
• Sampling Permutations for Shapley Value Estimation
• Sparsity in Deep Learning: Pruning and growth for efficient inference\n and training in neural networks
• Improving KernelSHAP: Practical Shapley Value Estimation via Linear Regression
• Pruning neural networks without any data by iteratively conserving synaptic flow
• Shapley Value as Principled Metric for Structured Network Pruning
• SNIP: Single-shot Network Pruning based on Connection Sensitivity
• Antithetic and Monte Carlo kernel estimators for partial rankings
• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
• A Unified Approach to Interpreting Model Predictions
• Learning both Weights and Connections for Efficient Neural Networks
• Bounding the Estimation Error of Sampling-based Shapley Value Approximation
• Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)

Cited by

• SHARe-KAN: Post-Training Vector Quantization for Cache-Resident KAN Inference

Related

• Splines-Based Feature Importance in Kolmogorov-Arnold Networks: A Framework for Supervised Tabular Data Dimensionality Reduction
• Kolmogorov--Arnold Networks for Small Language Models
• Kolmogorov-Arnold Networks: A Critical Assessment of Claims, Performance, and Practical Viability
• Softly Symbolifying Kolmogorov-Arnold Networks
• KAA: Kolmogorov-Arnold Attention for Enhancing Attentive Graph Neural Networks