Macro F1 and Macro F1

2019/11/08 by Juri Opitz, Opitz, Juri, Sebastian Burst +1 · 13 citations
Computer Science · #FOS: Computer and information sciences #Imbalanced Data Classification Techniques #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Spam and Phishing Detection #Text and Document Classification Technologies

paper · pdf · doi:10.48550/arxiv.1911.03347

openalex publication_date 2019/11/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The 'macro F1' metric is frequently used to evaluate binary, multi-class and multi-label classification problems. Yet, we find that there exist two different formulas to calculate this quantity. In this note, we show that only under rare circumstances the two computations can be considered equivalent. More specifically, one formula well 'rewards' classifiers which produce a skewed error type distribution. In fact, the difference in outcome of the two computations can be as high as 0.5. The two computations may not only diverge in their scalar result but can also lead to different classifier rankings.

Citations

• A systematic analysis of performance measures for classification tasks

Cited by

• Hallucination Detection for LLM-based Text-to-SQL Generation via Two-Stage Metamorphic Testing
• Understanding Trust Toward Human versus AI-generated Health Information through Behavioral and Physiological Sensing
• SweetDeep: A Wearable AI Solution for Real-Time Non-Invasive Diabetes Screening
• Fusion or Confusion? Assessing the impact of visible-thermal image fusion for automated wildlife detection
• E-Test: E'er-Improving Test Suites
• Deep semi-supervised approach based on consistency regularization and similarity learning for weeds classification
• MEC³O: Multi-Expert Consensus for Code Time Complexity Prediction
• Sensei: Self-Supervised Sensor Name Segmentation
• An AutoML Framework using AutoGluonTS for Forecasting Seasonal Extreme Temperatures
• LLM-OREF: An Open Relation Extraction Framework Based on Large Language Models
• Aegis: Automated Error Generation and Attribution for Multi-Agent Systems
• Joint Modeling of Entities and Discourse Relations for Coherence Assessment
• Deep Learning-based Animal Behavior Analysis: Insights from Mouse Chronic Pain Models

Related

• Consistent algorithms for multi-label classification with macro-at-k metrics
• Thresholding Classifiers to Maximize F1 Score
• Why only Micro-F1? Class Weighting of Measures for Relation Classification
• The MCC-F1 curve: a performance evaluation technique for binary classification
• sigmoidF1: A Smooth F1 Score Surrogate Loss for Multilabel Classification