The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation
2020/01/02 by Davide Chicco, Giuseppe Jurman · 95 citations
Computer Science · #Data Mining Algorithms and Applications #Imbalanced Data Classification Techniques #Text and Document Classification Technologies
paper · pdf · doi:10.1186/s12864-019-6413-7
openalex publication_date 2020/01/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/31
Abstract
Abstract Background To evaluate binary classifications and their confusion matrices, scientific researchers can employ several statistical rates, accordingly to the goal of the experiment they are investigating. Despite being a crucial issue in machine learning, no widespread consensus has been reached on a unified elective chosen measure yet. Accuracy and F 1 score computed on confusion matrices have been (and still are) among the most popular adopted metrics in binary classification tasks. However, these statistical measures can dangerously show overoptimistic inflated results, especially on imbalanced datasets. Results The Matthews correlation coefficient (MCC), instead, is a more reliable statistical rate which produces a high score only if the prediction obtained good results in all of the four confusion matrix categories (true positives, false negatives, true negatives, and false positives), proportionally both to the size of positive elements and the size of negative elements in the dataset. Conclusions In this article, we show how MCC produces a more informative and truthful score in evaluating binary classifications than accuracy and F 1 score, by first explaining the mathematical properties, and then the asset of MCC in six synthetic use cases and in a real genomics scenario. We believe that the Matthews correlation coefficient should be preferred to accuracy and F 1 score in evaluating binary classification tasks by all scientific communities.
Citations
Cited by
- Structured Estimation of Heterogeneous Time Series
- SATISFY EASY LADDERS: accessible machine learning to facilitate analysis of larger sensitive text data samples in criminology
- Beyond binary: a machine-learning classification of childhood COVID-19 vaccination intentions using behavioural data
- GL4SDA: Predicting snoRNA-disease associations using GNNs and LLM embeddings
- EnAcrPred : A robust ensemble machine learning framework for identifying anti‐ CRISPR proteins
- Phi, Fei, Fo, Fum: Effect Sizes for Categorical Data That Use the Chi-Squared Statistic
- Dos and Don'ts of Machine Learning in Computer Security
- AliFilter: a machine learning approach to alignment filtering
- Kernel Density–Encoded MALDI-TOF Spectra with LightGBM Machine Learning for Multiplex Resistance Prediction in Klebsiella pneumoniae
- The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation
- Stay Tuned: Improving Sentiment Analysis and Stance Detection Using Large Language Models
- MediaMatch : Prediction of Bacterial Growth on Different Culture Media Using the XGBoost Algorithm
- Exploring Spiking Neural Networks for Binary Classification in Multivariate Time Series at the Edge
- Construction safety predictions with multi-head attention graph and sparse accident networks
- A Multi-dimensional Semantic Surprise Framework Based on Low-Entropy Semantic Manifolds for Fine-Grained Out-of-Distribution Detection
- Enhanced lithological mapping in arid crystalline regions using explainable AI and multi-spectral remote sensing data
- ROFI: A Deep Learning-Based Ophthalmic Sign-Preserving and Reversible Patient Face Anonymizer
- Progressive growing of Generative Adversarial Networks for improving data augmentation and skin cancer diagnosis
- A Digital Pheromone-Based Approach for In/Out-of-Control Classification
- Determination of bioavailable arsenic threshold and validation of modeled permissible total arsenic in paddy soil using machine learning
- Landscape of mobile genetic elements and their antibiotic resistance cargo in prokaryotic genomes
- C-Net: A Reliable Convolutional Neural Network for Biomedical Image Classification
- A multi-layered embedded intrusion detection framework for programmable logic controllers
- The Subgroup Imperative: Chest Radiograph Classifier Generalization Gaps in Patient, Setting, and Pathology Subgroups
- Attention-Enhanced Prototypical Learning for Few-Shot Infrastructure Defect Segmentation
- Brain functional connectivity, but not neuroanatomy, captures the interrelationship between sex and gender in preadolescents
- Machine learning tool for wave overtopping prediction based on the safety-operability ratio
- Humanly Certifying Superhuman Classifiers
- CAJAL enables analysis and integration of single-cell morphological data using metric geometry
- Non-Collaborative User Simulators for Tool Agents
- Development and internal validation of the Cleveland Clinic Bleeding Model to predict major bleeding risk at admission in medical inpatients
- AI-Enabled Crater-Based Navigation for Lunar Mapping
- Metapredict enables accurate disorder prediction across the Tree of Life
- A Closer Look at Classification Evaluation Metrics and a Critical Reflection of Common Evaluation Practice
- Belief Coevolution in a Social Network of Generalist and Specialist Large Language Models
- Domain-adaptive matching bridges synthetic and in vivo neural dynamics for neural circuit connectivity inference
- Brain Control of a Computer Cursor for Online Target Selection - A Non-Invasive BCI for Continuous Movement Decoding
- The State of Peptide Detectability in Computational Proteomics and Guidelines for AI Applications
- Semi-Supervised Rotation Forest
- KDLSQ-BERT: A Quantized Bert Combining Knowledge Distillation with Learned Step Size Quantization
- Autism Spectrum Disorder Detection Using Prominent Connectivity Features from Electroencephalography
- A data-driven approach to customer lifetime value prediction using probability and machine learning models
- Multiple Threshold‐Selection Methods Are Needed to Binarise Species Distribution Model Predictions
- Maximum entropy networks for large scale social network node analysis
- Multi-scale modeling of habitat suitability and human-induced risk for the critically endangered white-rumped vulture (Gyps bengalensis) in South and Southeast Asia
- Knowledge and Awareness of Performance Metrics in Dental Image Analysis With Artificial Intelligence: A Cross-Sectional Survey of Researchers and Clinicians
- Identifying Adaptive Footprints in the Presence of Demographic Uncertainty
- Interpretable Early Failure Detection via Machine Learning and Trace Checking-based Monitoring
- Evaluation of performance measures in predictive artificial intelligence models to support medical decisions: overview and guidance
- Reinforcement learning-based dynamic ensemble for missense variant effect prediction and tiered prioritization of VUS
- A survey on deep learning approaches for breast cancer diagnosis
- Anomaly Detection in Industrial Control Systems Based on Cross-Domain Representation Learning
- Scaling to Multimodal and Multichannel Heart Sound Classification with Synthetic and Augmented Biosignals
- Learning Majority-to-Minority Transformations with MMD and Triplet Loss for Imbalanced Classification
- Software Vulnerability Detection via Deep Learning over Disaggregated Code Graph Representation
- Synthetic Data Generation with Lorenzetti for Time Series Anomaly Detection in High-Energy Physics Calorimeters
- An Ethically Grounded LLM-Based Approach to Insider Threat Synthesis and Detection
- ADMETlab 2.0: an integrated online platform for accurate and comprehensive predictions of ADMET properties
- STM-Graph: A Python Framework for Spatio-Temporal Mapping and Graph Neural Network Predictions
- Clinically confirmed cohort reveals antioxidant genetic polymorphisms as potential susceptibility factors for long COVID after mild or asymptomatic COVID-19
- Machine-learning competition to grade EEG background patterns in newborns with hypoxic-ischaemic encephalopathy
- Mapping Toxic Comments Across Demographics: A Dataset from German Public Broadcasting
- Exploration of read-across-based predictions for unveiling the risk of FDA-labeled drug-induced cardiotoxicity (DICT) rank: A careful classification of the most extensive reference list of human drugs
- Harmonized Gradient Descent for Class Imbalanced Data Stream Online Learning
- DepressLLM: Interpretable domain-adapted language model for depression detection from real-world narratives
- Impacts of major floods on new human respiratory health symptoms in indoor environments
- Machine Learning‐Guided Repositioning of a SARS‐CoV‐2‐Targeting Molecular Series as Cruzain Inhibitors
- Hybrid Computational Framework Integrating Ensemble Learning, Molecular Docking, and Dynamics for Predicting Antimalarial Efficacy of Malaria Box Compounds
- Honest and Reliable Evaluation and Expert Equivalence Testing of Automated Neonatal Seizure Detection
- Explainable Deep Neural Network for Multimodal ECG Signals: Intermediate vs Late Fusion
- GeHirNet: A Gender-Aware Hierarchical Model for Voice Pathology Classification
- A Thickness Sensitive Vessel Extraction Framework for Retinal and Conjunctival Vascular Tortuosity Analysis
- Multi-output Deep-Supervised Classifier Chains for Plant Pathology
- Comparative study of ML models for IIoT intrusion detection: impact of data preprocessing and balancing
- CogME: A Cognition-Inspired Multi-Dimensional Evaluation Metric for Story Understanding
- Evaluating Generative AI Tools for Personalized Offline Recommendations: A Comparative Study
- A Graph-in-Graph Learning Framework for Drug-Target Interaction Prediction
- Gender Fairness of Machine Learning Algorithms for Pain Detection
- Toward Realistic Evaluations of Just-In-Time Vulnerability Prediction
- A novel attention mechanism for noise-adaptive and robust segmentation of microtubules in microscopy images
- MalBERT: Using Transformers for Cybersecurity and Malicious Software Detection
- Area-based epigraph and hypograph indices for functional outlier detection
- Word stress in self-supervised speech models: A cross-linguistic comparison
- Machine learning can predict survival of patients with heart failure from serum creatinine and ejection fraction alone
- Performance Analysis of a Foreground Segmentation Neural Network Model
- Absolute Evaluation Measures for Machine Learning: A Survey
- LARP: Learner-Agnostic Robust Data Prefiltering
- Benchmarking Unsupervised Strategies for Anomaly Detection in Multivariate Time Series
- The Syntactic Acceptability Dataset (Preview): A Resource for Machine Learning and Linguistic Analysis of English
- Metrics for Multi-Class Classification: an Overview
- Critical Appraisal of Fairness Metrics in Clinical Predictive AI
- Fair for a few: Improving Fairness in Doubly Imbalanced Datasets
- TransBind allows precise detection of DNA-binding proteins and residues using language models and deep learning
- Distribución espacio-temporal de Eichhornia crassipes (Mart.) Solms a través de teledetección en laguna La Turbina, Cuba
- TabulaTime: Novel multimodal deep learning for Acute Coronary Syndrome prediction through environmental and clinical data integration
Related