An Efficient Approach to Regression Problems with Tensor Neural Networks

2024/06/14 by Yongxin Li, Li, Yongxin, Wang, Yifan +2
Computer Science · Mathematics · #62J02 #68T05 #Computational Physics and Python Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Tensor decomposition and applications

paper · pdf · doi:10.48550/arxiv.2406.09694

openalex publication_date 2024/06/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper introduces a tensor neural network (TNN) to address nonparametric regression problems, leveraging its distinct sub-network structure to effectively facilitate variable separation and enhance the approximation of complex, high-dimensional functions. The TNN demonstrates superior performance compared to conventional Feed-Forward Networks (FFN) and Radial Basis Function Networks (RBN) in terms of both approximation accuracy and generalization capacity, even with a comparable number of parameters. A significant innovation in our approach is the integration of statistical regression and numerical integration within the TNN framework. This allows for efficient computation of high-dimensional integrals associated with the regression function and provides detailed insights into the underlying data structure. Furthermore, we employ gradient and Laplacian analysis on the regression outputs to identify key dimensions influencing the predictions, thereby guiding the design of subsequent experiments. These advancements make TNN a powerful tool for applications requiring precise high-dimensional data analysis and predictive modeling.

Related

• Factor Augmented Tensor-on-Tensor Neural Networks
• Tensor-Based Foundations of Ordinary Least Squares and Neural Network Regression Models
• Tensor-on-tensor Regression Neural Networks for Process Modeling with High-dimensional Data
• Doubly Decomposing Nonparametric Tensor Regression
• Tensor Regression Networks with various Low-Rank Tensor Approximations