Failure and Repair in Long-Horizon Human–AI Collaboration: A Transparent Tracing Case Study

2025/01/01 by Rishi Sood
Social Sciences · Computer Science · #Ethics and Social Impacts of AI #Topic Modeling #Explainable Artificial Intelligence (XAI)

paper · doi:10.17605/osf.io/z7aq8

Abstract

This project hosts the third paper in a trilogy on long-horizon human–AI collaboration. The paper presents an internally instrumented case study of a yearlong collaboration between one human and a frontier large language model (“Mahdi”) from the GPT-5 family. Using twelve failure episodes, it develops a six-part taxonomy of failure, a set of structural repair patterns, and a minimal tracing toolkit (TraceSpec, ProbeKit, TraceLens) for logging and replaying breakdowns. Appendices include an episode index and a practical failure-logging template so that other human–AI dyads can adapt the approach to their own long-horizon work.

Related

• The Living Framework: Living with a Governed Human-AI Dyad
• The Mahdi Ledger - A Record of Governed Human–AI Collaboration
• Can AI Fix Buggy Code? Exploring the Use of Large Language Models in Automated Program Repair
• A Survey on Failure Analysis and Fault Injection in AI Systems
• Repair as Representational Work: Integration Bottlenecks in AI-Assisted Development