Riemannian Optimization on Tree Tensor Networks with Application in Machine Learning

2025/07/29 by Willner, Marius, Trenti, Marco, Lebiedz, Dirk
#15A69 #53C20 #65K10 #FOS: Computer and information sciences #FOS: Mathematics #FOS: Physical sciences #Machine Learning (cs.LG) #Optimization and Control (math.OC) #Other Condensed Matter (cond-mat.other)

paper · doi:10.48550/arxiv.2507.21726

Abstract

Tree tensor networks (TTNs) are widely used in low-rank approximation and quantum many-body simulation. In this work, we present a formal analysis of the differential geometry underlying TTNs. Building on this foundation, we develop efficient first- and second-order optimization algorithms that exploit the intrinsic quotient structure of TTNs. Additionally, we devise a backpropagation algorithm for training TTNs in a kernel learning setting. We validate our methods through numerical experiments on a representative machine learning task.

Citations

• Model reduction on manifolds: A differential geometric framework
• On Differential Geometric Formulations of Slow Invariant Manifold Computation: Geodesic Stretching and Flow Curvature
• Learning with tree-based tensor formats
• Optimization on the Hierarchical Tucker manifold - applications to\n tensor completion
• A literature survey of low-rank tensor approximation techniques

Related

• Tensor Network States with Low-Rank Tensors
• Second-order optimization for tensors with fixed tensor-train rank
• Tensor Network Estimation of Distribution Algorithms
• Regularized second-order optimization of tensor-network Born machines
• Structured Low-Rank Tensor Learning