Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing
2020/01/03 by Inioluwa Deborah Raji, Andrew Smart, Raji, Inioluwa Deborah +15 · 45 citations
Computer Science · Social Sciences · #Blockchain Technology Applications and Security #Computers and Society (cs.CY) #Ethics and Social Impacts of AI #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences
paper · pdf · doi:10.48550/arxiv.2001.00973
openalex publication_date 2020/01/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Abstract
Rising concern for the societal implications of artificial intelligence systems has inspired a wave of academic and journalistic literature in which deployed systems are audited for harm by investigators from outside the organizations deploying the algorithms. However, it remains challenging for practitioners to identify the harmful repercussions of their own systems prior to deployment, and, once deployed, emergent issues can become difficult or impossible to trace back to their source. In this paper, we introduce a framework for algorithmic auditing that supports artificial intelligence system development end-to-end, to be applied throughout the internal organization development lifecycle. Each stage of the audit yields a set of documents that together form an overall audit report, drawing on an organization's values or principles to assess the fit of decisions made throughout the process. The proposed auditing framework is intended to contribute to closing the accountability gap in the development and deployment of large-scale artificial intelligence systems by embedding a robust process to ensure audit integrity.
Citations
Cited by
- Security and Privacy in Agentic AI: Grand Challenges and Future Directions
- Toward Secure and Compliant AI: Organizational Standards and Protocols for NLP Model Lifecycle Management
- Stability of AI Governance Systems: A Coupled Dynamics Model of Public Trust and Social Disruptions
- Beyond Context: Large Language Models Failure to Grasp Users Intent
- Evaluation of AI Ethics Tools in Language Models: A Developers' Perspective Case Stud
- Insured Agents: A Decentralized Trust Insurance Mechanism for Agentic Economy
- Epistemic Substitution: How Grokipedia's AI-Generated Encyclopedia Restructures Authority
- Who Evaluates AI's Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluations
- Fair and Explainable Credit-Scoring under Concept Drift: Adaptive Explanation Frameworks for Evolving Populations
- EvalCards: A Framework for Standardized Evaluation Reporting
- Trustworthy Quantum Machine Learning: A Roadmap for Reliability, Robustness, and Security in the NISQ Era
- Validity Is What You Need
- Auditing LLM Editorial Bias in News Media Exposure
- Accountability in Government Use of <scp>AI</scp> : Citizen Concerns and Preferences
- Why Public Service AI Governance Frameworks Risk Failing in the Age of General-Purpose AI: Lessons from Policing
- The Human Utility Factor: A Computable Welfare Metric That Reframes AI Governance as a Constrained Optimisation Problem
- Metacognition Should Be the Scientific Framework for Bounded and Effective Self-Governance in Generative AI
- Exploring the Intersection of AI, Language, and Law: A Bibliometric Analysis
- Epistemic welfare and public service media's algorithmic recommender systems: A theoretical framework, operationalization and relevance for governance
- Black Box Absorption: LLMs Undermining Innovative Ideas
- AI of the People, by the People, for the People: A Social Choice Approach to Collective Control of Artificial Intelligence
- Measuring What Matters: The AI Pluralism Index
- A global log for medical AI
- "I Don't Think RAI Applies to My Model'' -- Engaging Non-champions with Sticky Stories for Responsible AI Work
- Does AI Coaching Prepare us for Workplace Negotiations?
- Holding AI to Account: Challenges for the Delivery of Trustworthy AI in Healthcare
- Scaling, Lock-In, and Proxy Compliance: A Political Economy of Responsible AI
- Bureaucratic Silences: What the Canadian AI Register Reveals, Omits, and Obscures
- Public acceptance of various ways to use and treat AI: A predictive moral framework
- From moral panic to pragmatic governance: reframing AI’s societal impacts in employment, education, and ethics
- Right-to-Override for Critical Urban Control Systems: A Deliberative Audit Method for Buildings, Power, and Transport
- Collective Recourse for Generative Urban Visualizations
- AI Harmonics: a human-centric and harms severity-adaptive AI risk assessment framework
- Audits Under Resource, Data, and Access Constraints: Scaling Laws For Less Discriminatory Alternatives
- "She was useful, but a bit too optimistic": Augmenting Design with Interactive Virtual Personas
- What Makes AI Applications Acceptable or Unacceptable? A Predictive Moral Framework
- Bridging Research Gaps Between Academic Research and Legal Investigations of Algorithmic Discrimination
- Ethics as a Service: A Pragmatic Operationalisation of AI Ethics
- The AI-Fraud Diamond: A Novel Lens for Auditing Algorithmic Deception
- Bias is a Math Problem, AI Bias is a Technical Problem: 10-year Literature Review of AI/LLM Bias Research Reveals Narrow [Gender-Centric] Conceptions of 'Bias', and Academia-Industry Gap
- TechOps: Technical Documentation Templates for the AI Act
- Towards Transparent Ethical AI: A Roadmap for Trustworthy Robotic Systems
- A Survey on Data Security in Large Language Models
- Mapping Stakeholder Needs to Multi-Sided Fairness in Candidate Recommendation for Algorithmic Hiring
- Exposing the Illusion of Fairness: Auditing Vulnerabilities to Distributional Manipulation Attacks
Related