Oversampling for Imbalanced Learning Based on K-Means and SMOTE

2017/11/30 by Georgios Douzas, Fernando Bacao, Fernando Bação +1 · 1,195 citations
Computer Science · Engineering · Mathematics · #Algorithm #Anomaly Detection Techniques and Applications #Artificial intelligence #Bandwidth (computing) #Class (philosophy) #Classifier (UML) #Cluster analysis #Computer science #Data mining #Electricity Theft Detection Techniques #Heuristic #Imbalanced Data Classification Techniques #Machine learning #Noise (video) #Oversampling #Python (programming language) #cs.LG #stat.ML

paper · pdf · doi:10.1016/j.ins.2018.06.056

published in Information Sciences 465, 1-20 (Elsevier BV) · 19 pages, 8 figures

openalex created_date 2017/11/10 · arxiv created 2017/12/12 · openalex publication_date 2018/06/28 · arxiv updated 2020/03/06 · openalex updated_date 2026/08/05

Abstract

Learning from class-imbalanced data continues to be a common and challenging problem in supervised learning as standard classification algorithms are designed to handle balanced class distributions. While different strategies exist to tackle this problem, methods which generate artificial data to achieve a balanced class distribution are more versatile than modifications to the classification algorithm. Such techniques, called oversamplers, modify the training data, allowing any classifier to be used with class-imbalanced datasets. Many algorithms have been proposed for this task, but most are complex and tend to generate unnecessary noise. This work presents a simple and effective oversampling method based on k-means clustering and SMOTE oversampling, which avoids the generation of noise and effectively overcomes imbalances between and within classes. Empirical results of extensive experiments with 71 datasets show that training data oversampled with the proposed method improves classification results. Moreover, k-means SMOTE consistently outperforms other popular oversampling methods. An implementation is made available in the python programming language.

Cited by

• A Comprehensive Analysis of Synthetic Minority Oversampling Technique (SMOTE) for handling class imbalance
• RUBRIC: Realism--Utility Balanced Ranking for Imbalanced Classification
• Data Balancing Strategies: A Systematic Survey of Resampling and Augmentation Methods
• Enhancing the classification of isolated theropod teeth using machine learning: a comparative study
• Envelope Imbalance Learning Algorithm based on Multilayer Fuzzy C-means Clustering and Minimum Interlayer discrepancy
• Geometric SMOTE for regression
• A Novel Resampling Technique for Imbalanced Dataset Optimization
• FA(IR)2MA-GLVQ – A hidden-feature-bias mitigation approach for fairness in classification learning based on generalized matrix learning vector quantization
• SetConv: A New Approach for Learning from Imbalanced Data
• Identification of human mental workload levels in a language comprehension task with imbalance neurophysiological data
• Towards Balanced Behavior Cloning from Imbalanced Datasets
• Handling Imbalanced Data: A Case Study for Binary Class Problems
• DRAST -- A Deep Learning and AST Based Approach for Bug Localization
• When Deep Learning Meets Information Retrieval-based Bug Localization: A Survey
• X-Ray Spectra and Multiwavelength Machine Learning Classification for Likely Counterparts to Fermi 3FGL Unassociated Sources
• Sampling - Variational Auto Encoder - Ensemble: In the Quest of Explainable Artificial Intelligence
• Enhancing Metabolic Syndrome Prediction with Hybrid Data Balancing and Counterfactuals
• Automatic Grading of Stroke Symptoms for Rapid Assessment Using Optimized Machine Learning and 4-Limb Kinematics: Clinical Validation Study. [europepmc]
• Prediction of neonatal deaths in NICUs: development and validation of machine learning models. [europepmc]
• Experimental Study and Comparison of Imbalance Ensemble Classifiers with Dynamic Selection Strategy. [europepmc]
• Prediction of In-Hospital Cardiac Arrest Using Shallow and Deep Learning. [europepmc]
• A Hybrid Method to Predict Postoperative Survival of Lung Cancer Using Improved SMOTE and Adaptive SVM. [europepmc]
• Tourist Experiences Recommender System Based on Emotion Recognition with Wearable Data. [europepmc]
• An Explainable Machine Learning Approach for COVID-19's Impact on Mood States of Children and Adolescents during the First Lockdown in Greece. [europepmc]
• Risk Prediction of Major Adverse Cardiovascular Events Occurrence Within 6 Months After Coronary Revascularization: Machine Learning Study. [europepmc]
• Property space mapping of Pseudomonas aeruginosa permeability to small molecules. [europepmc]
• Melanoma Detection Using XGB Classifier Combined with Feature Extraction and K-Means SMOTE Techniques. [europepmc]
• Ensemble stacking rockburst prediction model based on Yeo-Johnson, K-means SMOTE, and optimal rockburst feature dimension determination. [europepmc]
• A Machine Learning-Based Water Potability Prediction Model by Using Synthetic Minority Oversampling Technique and Explainable AI. [europepmc]
• Prediction of successful aging using ensemble machine learning algorithms. [europepmc]
• Predicting 180-day mortality for women with ovarian cancer using machine learning and patient-reported outcome data. [europepmc]
• A Proposed Framework for Early Prediction of Schistosomiasis. [europepmc]
• A hybrid sampling algorithm combining synthetic minority over-sampling technique and edited nearest neighbor for missed abortion diagnosis. [europepmc]
• Analyzing the effect of data preprocessing techniques using machine learning algorithms on the diagnosis of COVID-19. [europepmc]
• Comparative study of multiple machine learning algorithms for risk level prediction in goaf. [europepmc]
• Artificial Intelligence and Machine Learning Methods to Evaluate Cardiotoxicity following the Adverse Outcome Pathway Frameworks. [europepmc]
• Pruning-based oversampling technique with smoothed bootstrap resampling for imbalanced clinical dataset of Covid-19. [europepmc]
• Predicting diabetes in adults: identifying important features in unbalanced data over a 5-year cohort study using machine learning algorithm. [europepmc]
• Machine Learning-Based Radiomics Analysis for Identifying KRAS Mutations in Non-Small-Cell Lung Cancer from CT Images: Challenges, Insights and Implications. [europepmc]
• A review of machine learning methods for imbalanced data challenges in chemistry. [europepmc]
• The impact of online food delivery applications on dietary pattern disruption in the Arab region. [europepmc]
• An improved SMOTE algorithm for enhanced imbalanced data classification by expanding sample generation space. [europepmc]

Related

• Stop Oversampling for Class Imbalance Learning: A Critical Review
• Kernel-Based Enhanced Oversampling Method for Imbalanced Classification
• Clustering and Learning from Imbalanced Data
• Deep Learning Meets Oversampling: A Learning Framework to Handle Imbalanced Classification
• WOTBoost: Weighted Oversampling Technique in Boosting for imbalanced learning