Automating AI Failure Tracking: Semantic Association of Reports in AI Incident Database

2025/07/31 by Russo, Diego, Gian Marco Orlando, Orlando, Gian Marco +4
Computer Science · Decision Sciences · #Artificial Intelligence (cs.AI) #Computers and Society (cs.CY) #Data Quality and Management #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Information Retrieval (cs.IR) #Software Engineering Research

paper · pdf · doi:10.48550/arxiv.2507.23669

openalex publication_date 2025/07/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Artificial Intelligence (AI) systems are transforming critical sectors such as healthcare, finance, and transportation, enhancing operational efficiency and decision-making processes. However, their deployment in high-stakes domains has exposed vulnerabilities that can result in significant societal harm. To systematically study and mitigate these risk, initiatives like the AI Incident Database (AIID) have emerged, cataloging over 3,000 real-world AI failure reports. Currently, associating a new report with the appropriate AI Incident relies on manual expert intervention, limiting scalability and delaying the identification of emerging failure patterns. To address this limitation, we propose a retrieval-based framework that automates the association of new reports with existing AI Incidents through semantic similarity modeling. We formalize the task as a ranking problem, where each report-comprising a title and a full textual description-is compared to previously documented AI Incidents based on embedding cosine similarity. Benchmarking traditional lexical methods, cross-encoder architectures, and transformer-based sentence embedding models, we find that the latter consistently achieve superior performance. Our analysis further shows that combining titles and descriptions yields substantial improvements in ranking accuracy compared to using titles alone. Moreover, retrieval performance remains stable across variations in description length, highlighting the robustness of the framework. Finally, we find that retrieval performance consistently improves as the training set expands. Our approach provides a scalable and efficient solution for supporting the maintenance of the AIID.

Citations

• Pathway to Relevance: How Cross-Encoders Implement a Semantic Variant of BM25
• The AI Incident Database as an Educational Tool to Raise Awareness of AI Harms: A Classroom Exploration of Efficacy, Limitations, & Future Improvements
• "Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practice
• In Defense of Cross-Encoders for Zero-Shot Retrieval
• Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database
• ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT
• Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation
• MPNet: Masked and Permuted Pre-training for Language Understanding
• Dense Passage Retrieval for Open-Domain Question Answering
• Document Ranking with a Pretrained Sequence-to-Sequence Model
• DC-BERT: Decoupling Question and Document for Efficient Contextual Encoding
• MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• MultiQA: An Empirical Investigation of Generalization and Transfer in Reading Comprehension
• Document Expansion by Query Prediction

Related

• Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database
• A taxonomic system for failure cause analysis of open source AI incidents
• Lessons for Editors of AI Incidents from the AI Incident Database
• Indexing AI Risks with Incidents, Issues, and Variants
• Output Scouting: Auditing Large Language Models for Catastrophic Responses