Interpretable Bayesian Tensor Network Kernel Machines with Automatic Rank and Feature Selection

2025/07/15 by Kilic, Afra, Kim Batselier, Batselier, Kim · 1 citation
Computer Science · Mathematics · #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Tensor decomposition and applications

paper · pdf · doi:10.48550/arxiv.2507.11136

openalex publication_date 2025/07/15 · openalex created_date 2025/10/08 · openalex updated_date 2026/07/28

Abstract

Tensor Network (TN) Kernel Machines speed up model learning by representing parameters as low-rank TNs, reducing computation and memory use. However, most TN-based Kernel methods are deterministic and ignore parameter uncertainty. Further, they require manual tuning of model complexity hyperparameters like tensor rank and feature dimensions, often through trial-and-error or computationally costly methods like cross-validation. We propose Bayesian Tensor Network Kernel Machines, a fully probabilistic framework that uses sparsity-inducing hierarchical priors on TN factors to automatically infer model complexity. This enables automatic inference of tensor rank and feature dimensions, while also identifying the most relevant features for prediction, thereby enhancing model interpretability. All the model parameters and hyperparameters are treated as latent variables with corresponding priors. Given the Bayesian approach and latent variable dependencies, we apply a mean-field variational inference to approximate their posteriors. We show that applying a mean-field approximation to TN factors yields a Bayesian ALS algorithm with the same computational complexity as its deterministic counterpart, enabling uncertainty quantification at no extra computational cost. Experiments on synthetic and real-world datasets demonstrate the superior performance of our model in prediction accuracy, uncertainty quantification, interpretability, and scalability.

Citations

• Quantized Fourier and Polynomial Features for more Expressive Tensor Network Models
• Large-Scale Learning with Fourier Features and Tensor Decompositions
• Bayesian Deep Convolutional Networks with Many Channels are Gaussian\n Processes
• Deep Convolutional Networks as shallow Gaussian Processes
• Scalable Gaussian Processes with Billions of Inducing Inputs via Tensor Train Decomposition
• Tensor Regression Meets Gaussian Processes
• Gaussian Quadrature for Kernel Features
• Variational Fourier features for Gaussian processes
• Tensor Network alternating linear scheme for MIMO Volterra system identification
• Bayesian Tensor Regression
• Bayesian Sparse Tucker Models for Dimension Reduction and Tensor Completion
• Hilbert space methods for reduced-rank Gaussian process regression
• Equivariant and scale-free Tucker decomposition models
• Bayesian Conditional Tensor Factorizations for High-Dimensional Classification
• Tensor Decompositions and Applications
• Analysis of Individual Differences in Multidimensional Scaling Via an N-way Generalization of “Eckart-Young” Decomposition
• Some Mathematical Notes on Three-Mode Factor Analysis
• Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics

Cited by

• A Fully Probabilistic Tensor Network for Regularized Volterra System Identification

Related

• Tensor Methods for Generating Compact Uncertainty Quantification and Deep Learning Models
• Bayesian Tensorized Neural Networks with Automatic Rank Selection
• A Fully Probabilistic Tensor Network for Regularized Volterra System Identification
• Tensor Network Based Feature Learning Model
• The Bayesian Method of Tensor Networks