Classifying Hate: Legal and Ethical Evaluations of ML-Assisted Hate Crime Classification and Estimation in Sweden

2025/06/23 by Holli Sargeant, Hannes Waldetoft, Måns Magnusson · 1 voice
Computer Science · Social Sciences · #Hate Speech and Cyberbullying Detection #Law in Society and Culture

paper · pdf · doi:10.1145/3715275.3732016

openalex publication_date 2025/06/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/29

Abstract

Hate crimes, driven by biases against specific demographic groups, harm not only individuals but undermine the security, trust, and cohesion of entire communities.Accurately identifying such crimes remains a significant challenge due to under-reporting, limited training, and the complexity of determining bias motivations.In this paper, we analyze the results of a text classification model developed to improve the precision of hate crime statistics and identification in Sweden.Empirical results indicate the model outperforms traditional manual police classification of hate crimes, achieving higher precision across various crime types and regions.We further disaggregate performance to pinpoint persistent challenges and highlight categories where both human and machine decision-makers struggle.While the model focuses on statistical estimation rather than direct case-level decision-making, we discuss the broader implications of algorithmic transparency, accountability, and explainability.Ultimately, this research illustrates how transformer-based neural networks can responsibly bolster the detection and understanding of hate crimes, informing policies to better protect vulnerable communities.

Citations

• Fairness in Criminal Justice Risk Assessments: The State of the Art
• Prediction-powered estimators for finite population statistics in highly imbalanced textual data: Public hate crime estimation
• Uncovering Latent Memories: Assessing Data Leakage and Memorization Patterns in Frontier AI Models
• Evidence of What, for Whom? The Socially Contested Role of Algorithmic Bias in a Predictive Policing Tool
• Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models
• Extracting Training Data from Large Language Models
• Playing with Words at the National Library of Sweden -- Making a Swedish BERT
• Reporting the Unreported: Event Extraction for Analyzing the Local\n Representation of Hate Crimes
• Fairness in Criminal Justice Risk Assessments: The State of the Art
• Algorithmic decision making and the cost of fairness
• Fair prediction with disparate impact: A study of bias in recidivism\n prediction instruments
• Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments
• Inherent Trade-Offs in the Fair Determination of Risk Scores
• Inherent Trade-Offs in the Fair Determination of Risk Scores
• ‘We are all vulnerable’
• Does automation bias decision-making?

Discussions

• "Classifying Hate: Legal and Ethical Evaluations of ML-Assisted Hate Crime Classification and Estimation in Sweden" (w Hannes Waldetoft & @mansmag.bsky) studies where models succeed and struggle when [bsky, 1 points, 0 comments]

Related

• Indicators for characterising online hate speech and its automatic detection
• Harnessing Artificial Intelligence to Combat Online Hate: Exploring the Challenges and Opportunities of Large Language Models in Hate Speech Detection
• A Survey of Machine Learning Models and Datasets for the Multi-label Classification of Textual Hate Speech in English
• A Legal Approach to Hate Speech: Operationalizing the EU's Legal Framework against the Expression of Hatred as an NLP Task
• Evaluation of Hate Speech Detection Using Large Language Models and Geographical Contextualization