The Layered Structure of Tensor Estimation and its Mutual Information

2017/09/29 by Barbier, Jean, Macris, Nicolas, Miolane, Léo · 2 citations
#Disordered Systems and Neural Networks (cond-mat.dis-nn) #FOS: Computer and information sciences #FOS: Physical sciences #Information Theory (cs.IT) #Mathematical Physics (math-ph)

paper · doi:10.48550/arxiv.1709.10368

Abstract

We consider rank-one non-symmetric tensor estimation and derive simple formulas for the mutual information. We start by the order 2 problem, namely matrix factorization. We treat it completely in a simpler fashion than previous proofs using a new type of interpolation method developed in [1]. We then show how to harness the structure in "layers" of tensor estimation in order to obtain a formula for the mutual information for the order 3 problem from the knowledge of the formula for the order 2 problem, still using the same kind of interpolation. Our proof technique straightforwardly generalizes and allows to rigorously obtain the mutual information at any order in a recursive way.

Cited by

• An RDT based confirmation of Lehner's formula for Kronecker-Gaussian matrices
• Computational Thresholds in Multi-Modal Learning via the Spiked Matrix-Tensor Model

Related

• Tensor estimation with structured priors
• Statistical Limits for Finite-Rank Tensor Estimation
• Estimation of low-rank tensors via convex optimization
• Robust low-rank multilinear tensor approximation for a joint estimation of the multilinear rank and the loading matrices
• An algorithm for online tensor prediction