2015/07/30 by Shuchang Zhou, Yuxin Wu, Zhou, Shuchang +1
Computer Science · #Advanced Image Processing Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Image and Signal Denoising Methods #cs.CV
paper · pdf · doi:10.48550/arxiv.1507.08429
arxiv created 2015/07/30 · openalex publication_date 2015/07/30 · arxiv updated 2015/07/31 · openalex created_date 2016/06/24 · openalex updated_date 2026/07/28
In this paper we propose and study a technique to impose structural constraints on the output of a neural network, which can reduce amount of computation and number of parameters besides improving prediction accuracy when the output is known to approximately conform to the low-rankness prior. The technique proceeds by replacing the output layer of neural network with the so-called MLM layers, which forces the output to be the result of some Multilinear Map, like a hybrid-Kronecker-dot product or Kronecker Tensor Product. In particular, given an "autoencoder" model trained on SVHN dataset, we can construct a new model with MLM layer achieving 62% reduction in total number of parameters and reduction of ℓ2 reconstruction error from 0.088 to 0.004. Further experiments on other autoencoder model variants trained on SVHN datasets also demonstrate the efficacy of MLM layers.