Linear Support Vector Regression with Linear Constraints

2019/11/06 by Quentin Klopfenstein, Samuel Vaiter, Klopfenstein, Quentin +1
Computer Science · Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Face and Expression Recognition #Gaussian Processes and Bayesian Inference #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Statistical Methods and Inference #math.OC #stat.ML

paper · pdf · doi:10.48550/arxiv.1911.02306

arxiv created 2019/11/06 · openalex publication_date 2019/11/06 · arxiv updated 2019/11/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper studies the addition of linear constraints to the Support Vector Regression (SVR) when the kernel is linear. Adding those constraints into the problem allows to add prior knowledge on the estimator obtained, such as finding probability vector or monotone data. We propose a generalization of the Sequential Minimal Optimization (SMO) algorithm for solving the optimization problem with linear constraints and prove its convergence. Then, practical performances of this estimator are shown on simulated and real datasets with different settings: non negative regression, regression onto the simplex for biomedical data and isotonic regression for weather forecast.

Related

• Maximal Margin Distribution Support Vector Regression with coupled Constraints-based Convex Optimization
• Solving L1-regularized SVMs and related linear programs: Revisiting the effectiveness of Column and Constraint Generation
• The Planning-ahead SMO Algorithm
• Ordered Weighted Average Support Vector Regression
• MaxMin-L2-SVC-NCH: A Novel Approach for Support Vector Classifier Training and Parameter Selection