Multi-Merge Budget Maintenance for Stochastic Gradient Descent SVM Training

2018/06/26 by Sahar Qaadan, Tobias Glasmachers, Qaadan, Sahar +1
Computer Science · #Advanced Neural Network Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.1806.10179

openalex publication_date 2018/06/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Budgeted Stochastic Gradient Descent (BSGD) is a state-of-the-art technique for training large-scale kernelized support vector machines. The budget constraint is maintained incrementally by merging two points whenever the pre-defined budget is exceeded. The process of finding suitable merge partners is costly; it can account for up to 45% of the total training time. In this paper we investigate computationally more efficient schemes that merge more than two points at once. We obtain significant speed-ups without sacrificing accuracy.

Citations

• A Divide-and-Conquer Solver for Kernel Support Vector Machines

Related

• Speeding Up Budgeted Stochastic Gradient Descent SVM Training with Precomputed Golden Section Search
• Dual SVM Training on a Budget
• Soft Merging: A Flexible and Robust Soft Model Merging Approach for Enhanced Neural Network Performance
• Parallel Stochastic Gradient Descent with Sound Combiners
• The Kernelized Stochastic Batch Perceptron