Fully tensorial approach to hypercomplex neural networks

2024/06/29 by Niemczynowicz, Agnieszka, Kycia, Radosław Antoni
#15-04 #15A69 #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural and Evolutionary Computing (cs.NE)

paper · doi:10.48550/arxiv.2407.00449

Abstract

Fully tensorial theory of hypercomplex neural networks is given. It allows neural networks to use arithmetic based on arbitrary algebras. The key point is to observe that algebra multiplication can be represented as a rank three tensor and use this tensor in every algebraic operation. This approach is attractive for neural network libraries that support effective tensorial operations. It agrees with previous implementations for four-dimensional algebras.

Related

• KHNNs: hypercomplex neural networks computations via Keras using TensorFlow and PyTorch
• Tensor Neural Network and Its Numerical Integration
• Universal Approximation Theorem for Vector- and Hypercomplex-Valued Neural Networks
• Tensorial Neural Networks: Generalization of Neural Networks and Application to Model Compression
• Stable Tensor Neural Networks for Rapid Deep Learning