Large Scale Kernel Learning using Block Coordinate Descent

2016/02/17 by Stephen Tu, Tu, Stephen, Rebecca Roelofs +5 · 2 citations
Computer Science · Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #cs.LG #math.OC #stat.ML

paper · pdf · doi:10.48550/arxiv.1602.05310

arxiv created 2016/02/17 · arxiv updated 2016/02/18

Abstract

We demonstrate that distributed block coordinate descent can quickly solve kernel regression and classification problems with millions of data points. Armed with this capability, we conduct a thorough comparison between the full kernel, the Nyström method, and random features on three large classification tasks from various domains. Our results suggest that the Nyström method generally achieves better statistical accuracy than random features, but can require significantly more iterations of optimization. Lastly, we derive new rates for block coordinate descent which support our experimental findings when specialized to kernel methods.

Cited by

• Joker: Joint Optimization Framework for Lightweight Kernel Machines
• Exploiting Block Coordinate Descent for Cost-Effective LLM Model Training

Related

• A Simple Algorithm For Scaling Up Kernel Methods
• Joker: Joint Optimization Framework for Lightweight Kernel Machines
• Randomized Clustered Nystrom for Large-Scale Kernel Machines
• Distributed Coordinate Descent Method for Learning with Big Data
• A better convergence analysis of the block coordinate descent method for large scale machine learning