Recipe for Fast Large-scale SVM Training: Polishing, Parallelism, and more RAM!

2022/07/03 by Tobias Glasmachers, Glasmachers, Tobias
Computer Science · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Stochastic Gradient Optimization Techniques

paper · pdf · doi:10.48550/arxiv.2207.01016

openalex publication_date 2022/07/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Support vector machines (SVMs) are a standard method in the machine learning toolbox, in particular for tabular data. Non-linear kernel SVMs often deliver highly accurate predictors, however, at the cost of long training times. That problem is aggravated by the exponential growth of data volumes over time. It was tackled in the past mainly by two types of techniques: approximate solvers, and parallel GPU implementations. In this work, we combine both approaches to design an extremely fast dual SVM solver. We fully exploit the capabilities of modern compute servers: many-core architectures, multiple high-end GPUs, and large random access memory. On such a machine, we train a large-margin classifier on the ImageNet data set in 24 minutes.

Related

• Fast Support Vector Machines Using Parallel Adaptive Shrinking on Distributed Systems
• A Comparison of Support Vector Machines Training GPU-Accelerated Open Source Implementations
• Parallel Support Vector Machines in Practice
• Kernel machines that adapt to GPUs for effective large batch training
• Dual coordinate solvers for large-scale structural SVMs