The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models

2021/07/01 by Greg d'Eon, Jason d’Eon, d'Eon, Greg +5 · 2 citations
Computer Science · #Adversarial Robustness in Machine Learning #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2107.00758

openalex publication_date 2021/07/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Supervised learning models often make systematic errors on rare subsets of the data. When these subsets correspond to explicit labels in the data (e.g., gender, race) such poor performance can be identified straightforwardly. This paper introduces a method for discovering systematic errors that do not correspond to such explicitly labelled subgroups. The key idea is that similar inputs tend to have similar representations in the final hidden layer of a neural network. We leverage this structure by "shining a spotlight" on this representation space to find contiguous regions where the model performs poorly. We show that the spotlight surfaces semantically meaningful areas of weakness in a wide variety of existing models spanning computer vision, NLP, and recommender systems.

Cited by

• TRoVe: Discovering Error-Inducing Static Feature Biases in Temporal Vision-Language Models
• Active Slice Discovery in Large Language Models

Related

• What could go wrong? Discovering and describing failure modes in computer vision
• UDIS: Unsupervised Discovery of Bias in Deep Visual Recognition Models
• FairDropout: Using Example-Tied Dropout to Enhance Generalization of Minority Groups
• Learning from others' mistakes: Avoiding dataset biases without modeling them
• Say My Name: a Model's Bias Discovery Framework