Optimization on the Extended Tensor-Train Manifold with Shared Factors

2025/08/28 by Alexander Molozhavenko, Molozhavenko, Alexander, Maxim Rakhuba +1
Engineering · Mathematics · #15A23 #15A69 #53Z50 #65F15 #Elasticity and Material Modeling #FOS: Mathematics #Numerical Analysis (math.NA) #Tensor decomposition and applications

paper · pdf · doi:10.48550/arxiv.2508.20928

openalex publication_date 2025/08/28 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper studies tensors that admit decomposition in the Extended Tensor Train (ETT) format, with a key focus on the case where some decomposition factors are constrained to be equal. This factor sharing introduces additional challenges, as it breaks the multilinear structure of the decomposition. Nevertheless, we show that Riemannian optimization methods can naturally handle such constraints and prove that the underlying manifold is indeed smooth. We develop efficient algorithms for key Riemannian optimization components, including a retraction operation based on quasi-optimal approximation in the new format, as well as tangent space projection using automatic differentiation. Finally, we demonstrate the practical effectiveness of our approach through tensor approximation tasks and multidimensional eigenvalue problem.

Citations

• Low-rank matrix and tensor approximations: advancing efficiency of machine-learning interatomic potentials
• Riemannian preconditioned algorithms for tensor completion via tensor ring decomposition
• Automatic differentiation for Riemannian optimization on low-rank matrix and tensor-train manifolds
• T-Basis: a Compact Representation for Neural Networks
• Randomized Algorithms for Computation of Tucker decomposition and Higher Order SVD (HOSVD)
• Jacobi-Davidson method on low-rank matrix manifolds
• Tensor Ring Decomposition
• Tensor Methods and Recommender Systems
• Tensorizing Neural Networks
• Preconditioned low-rank Riemannian optimization for linear systems with tensor product structure
• Speeding-up Convolutional Neural Networks Using Fine-tuned\n CP-Decomposition
• A literature survey of low-rank tensor approximation techniques
• Tensor Decompositions and Applications
• Toward the Optimal Preconditioned Eigensolver: Locally Optimal Block Preconditioned Conjugate Gradient Method
• LoRA meets Riemannion: Muon Optimizer for Parametrization-independent Low-Rank Adapters
• TT-cross approximation for multidimensional arrays

Related

• Second-order optimization for tensors with fixed tensor-train rank
• Riemannian Tensor Completion with Side Information
• Note: low-rank tensor train completion with side information based on Riemannian optimization
• Automatic differentiation for Riemannian optimization on low-rank matrix and tensor-train manifolds
• Provable Tensor-Train Format Tensor Completion by Riemannian Optimization