Learning Deep Representations By Distributed Random Samplings

2013/12/16 by Zhang, Xiao-Lei
#FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.1312.4405

Abstract

In this paper, we propose an extremely simple deep model for the unsupervised nonlinear dimensionality reduction -- deep distributed random samplings, which performs like a stack of unsupervised bootstrap aggregating. First, its network structure is novel: each layer of the network is a group of mutually independent k-centers clusterings. Second, its learning method is extremely simple: the k centers of each clustering are only k randomly selected examples from the training data; for small-scale data sets, the k centers are further randomly reconstructed by a simple cyclic-shift operation. Experimental results on nonlinear dimensionality reduction show that the proposed method can learn abstract representations on both large-scale and small-scale problems, and meanwhile is much faster than deep neural networks on large-scale problems.

Related

• Learning Deep Representation Without Parameter Inference for Nonlinear Dimensionality Reduction
• Deep Distributed Random Samplings for Supervised Learning: An Alternative to Random Forests?
• Simple Deep Random Model Ensemble
• Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering
• Random Projection in Deep Neural Networks