2014/05/18 by Dhruv Mahajan, Mahajan, Dhruv, S. Sathiya Keerthi +3
Computer Science · #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Neural Networks and Applications #Stochastic Gradient Optimization Techniques #cs.LG
paper · pdf · doi:10.48550/arxiv.1405.4543
arxiv created 2014/05/18 · openalex publication_date 2014/05/18 · arxiv updated 2014/05/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
This paper concerns the distributed training of nonlinear kernel machines on Map-Reduce. We show that a re-formulation of Nyström approximation based solution which is solved using gradient based techniques is well suited for this, especially when it is necessary to work with a large number of basis points. The main advantages of this approach are: avoidance of computing the pseudo-inverse of the kernel sub-matrix corresponding to the basis points; simplicity and efficiency of the distributed part of the computations; and, friendliness to stage-wise addition of basis points. We implement the method using an AllReduce tree on Hadoop and demonstrate its value on a few large benchmark datasets.