2002/01/01 by Sebastian Risau-Gusman, Sebastián Risau-Gusman, Risau-Gusman, Sebastian +3
Computer Science · Engineering · Physics and Astronomy · #Control Systems and Identification #Disordered Systems and Neural Networks (cond-mat.dis-nn) #Evolutionary Algorithms and Applications #FOS: Physical sciences #Fault Detection and Control Systems #Neural Networks and Applications #cond-mat.dis-nn
paper · pdf · doi:10.48550/arxiv.cond-mat/0203315
26 pages, 10 figures
arxiv created 2002/03/14 · openalex publication_date 2002/03/14 · arxiv updated 2009/11/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Typical learning curves for Soft Margin Classifiers (SMCs) learning both realizable and unrealizable tasks are determined using the tools of Statistical Mechanics. We derive the analytical behaviour of the learning curves in the regimes of small and large training sets. The generalization errors present different decay laws towards the asymptotic values as a function of the training set size, depending on general geometrical characteristics of the rule to be learned. Optimal generalization curves are deduced through a fine tuning of the hyperparameter controlling the trade-off between the error and the regularization terms in the cost function. Even if the task is realizable, the optimal performance of the SMC is better than that of a hard margin Support Vector Machine (SVM) learning the same rule, and is very close to that of the Bayesian classifier.