Loss-Proportional Subsampling for Subsequent ERM

2013/06/07 by Paul Mineiro, Nikos Karampatziakis, Mineiro, Paul +1 · 2 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Imbalanced Data Classification Techniques #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms

paper · pdf · doi:10.48550/arxiv.1306.1840

openalex publication_date 2013/06/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose a sampling scheme suitable for reducing a data set prior to selecting a hypothesis with minimum empirical risk. The sampling only considers a subset of the ultimate (unknown) hypothesis set, but can nonetheless guarantee that the final excess risk will compare favorably with utilizing the entire original data set. We demonstrate the practical benefits of our approach on a large dataset which we subsample and subsequently fit with boosted trees.

Cited by

• Online Learning to Sample
• Local Uncertainty Sampling for Large-Scale Multi-Class Logistic Regression

Related

• First-Order Adaptive Sample Size Methods to Reduce Complexity of Empirical Risk Minimization
• Reweighting Improves Conditional Risk Bounds
• A subsampling approach for large data sets when the Generalised Linear Model is potentially misspecified
• Risk Minimization from Adaptively Collected Data: Guarantees for Supervised and Policy Learning
• Scaling laws for learning with real and surrogate data