Structured Variational D-Decomposition for Accurate and Stable Low-Rank Approximation

2025/06/10 by Ronald Katende, Katende, Ronald
Mathematics · Physics and Astronomy · Computer Science · #Tensor decomposition and applications #Model Reduction and Neural Networks #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2506.08535

Abstract

We introduce the D-decomposition, a non-orthogonal matrix factorization of the form A ≈ P D Q, where P ∈ ℝ^{n × k}, D ∈ ℝ^{k × k}, and Q ∈ ℝ^{k × n}. The decomposition is defined variationally by minimizing a regularized Frobenius loss, allowing control over rank, sparsity, and conditioning. Unlike algebraic factorizations such as LU or SVD, it is computed by alternating minimization. We establish existence and perturbation stability of the solution and show that each update has complexity O(n²k). Benchmarks against truncated SVD, CUR, and nonnegative matrix factorization show improved reconstruction accuracy on MovieLens, MNIST, Olivetti Faces, and gene expression matrices, particularly under sparsity and noise.

Citations

• Efficient Algorithms for Regularized Nonnegative Scale-invariant Low-rank Approximation Models
• A Validation Approach to Over-parameterized Matrix and Image Recovery
• Nonconvex Factorization and Manifold Formulations are Almost Equivalent in Low-rank Matrix Optimization
• Low-rank matrix recovery via iteratively reweighted least squares\n minimization
• Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization
• Sparse Principal Component Analysis
• Rank-revealing 𝑄𝑅 factorizations and the singular value decomposition

Related

• Core-Conditioned Regularized Matrix Tri-Factorization for High-Dimensional Structured Systems
• Structured Low-Rank Matrix Factorization: Global Optimality, Algorithms,\n and Applications
• In-depth Analysis of Low-rank Matrix Factorisation in a Federated Setting
• Efficient Structured Matrix Rank Minimization
• Accurate and fast matrix factorization for low-rank learning