Artificial intelligence in health care: accountability and safety
2020/02/25 by Ibrahim Habli, Tom Lawton, Zoe Porter +1 · 324 citations
Engineering · Health Professions · Medicine · Psychology · #Accountability #Artificial Intelligence in Healthcare and Education #Artificial intelligence #Blame #Clinical Reasoning and Diagnostic Skills #Computer science #Context (archaeology) #Disaster Response and Management #Engineering #Engineering ethics #Harm #Law #Medicine #Political science #Psychiatry #Psychology #Risk analysis (engineering) #Safety assurance #Social psychology
paper · pdf · doi:10.2471/blt.19.237487
published in Bulletin of the World Health Organization 98(4), 251-256 (World Health Organization)
openalex publication_date 2020/02/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06
Abstract
The prospect of patient harm caused by the decisions made by an artificial intelligence-based clinical tool is something to which current practices of accountability and safety worldwide have not yet adjusted. We focus on two aspects of clinical artificial intelligence used for decision-making: moral accountability for harm to patients; and safety assurance to protect patients against such harm. Artificial intelligence-based tools are challenging the standard clinical practices of assigning blame and assuring safety. Human clinicians and safety engineers have weaker control over the decisions reached by artificial intelligence systems and less knowledge and understanding of precisely how the artificial intelligence systems reach their decisions. We illustrate this analysis by applying it to an example of an artificial intelligence-based system developed for use in the treatment of sepsis. The paper ends with practical suggestions for ways forward to mitigate these concerns. We argue for a need to include artificial intelligence developers and systems safety engineers in our assessments of moral accountability for patient harm. Meanwhile, none of the actors in the model robustly fulfil the traditional conditions of moral accountability for the decisions of an artificial intelligence system. We should therefore update our conceptions of moral accountability in this context. We also need to move from a static to a dynamic model of assurance, accepting that considerations of safety are not fully resolvable during the design of the artificial intelligence system before the system has been deployed.
Cited by
- Anwendungen, Herausforderungen und ein vertrauenswürdiger Umgang mit künstlicher Intelligenz im Bereich Public Health
- The Trust in AI-Generated Health Advice (TAIGHA) Scale and Short Version (TAIGHA-S): Development and Validation Study
- Integrating Artificial Intelligence in Youth Mental Health Care: Advances, Challenges, and Future Directions
- Physicians’ required competencies in AI-assisted clinical settings: a systematic review
- Artificial Intelligence: Opportunities and Challenges for Public Administration
- Current state and prospects of artificial intelligence in allergy
- Why Chain of Thought Fails in Clinical Text Understanding
- RAGuard: A Novel Approach for in-context Safe Retrieval Augmented Generation for LLMs
- Accountability Framework for Healthcare AI Systems: Towards Joint Accountability in Decision Making
- Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications
- Student Pharmacists’ Perceptions of Artificial Intelligence and Machine Learning in Pharmacy Practice and Pharmacy Education
- Uncertainty Estimation on Graphs with Structure Informed Stochastic Partial Differential Equations
- Towards an Explanation Space to Align Humans and Explainable-AI Teamwork
- Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safety
- The ethical dimensions of utilizing Artificial Intelligence in palliative care
- Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI Extension
- Balancing risks and benefits of artificial intelligence in the health sector. [europepmc]
- Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI Extension. [europepmc]
- Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. [europepmc]
- Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. [europepmc]
- Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI Extension. [europepmc]
- Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. [europepmc]
- Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. [europepmc]
- Artificial Intelligence for Mental Health Care: Clinical Applications, Barriers, Facilitators, and Artificial Wisdom. [europepmc]
- Artificial Intelligence for Clinical Decision Support in Sepsis. [europepmc]
- Recommendations for the Development of Telemedicine in Poland Based on the Analysis of Barriers and Selected Telemedicine Solutions. [europepmc]
- From COVID-19 Pandemic to Patient Safety: A New "Spring" for Telemedicine or a Boomerang Effect? [europepmc]
- Quality of reporting of randomised controlled trials of artificial intelligence in healthcare: a systematic review. [europepmc]
- AI and machine learning ethics, law, diversity, and global impact. [europepmc]
- Principles for Health Information Collection, Sharing, and Use: A Policy Statement From the American Heart Association. [europepmc]
- Adopting and expanding ethical principles for generative artificial intelligence from military to healthcare. [europepmc]
- Artificial Intelligence in Health Care-Understanding Patient Information Needs and Designing Comprehensible Transparency: Qualitative Study. [europepmc]
- Ethical and regulatory challenges of AI technologies in healthcare: A narrative review. [europepmc]
- Twelve tips for addressing ethical concerns in the implementation of artificial intelligence in medical education. [europepmc]
- Ethical framework for artificial intelligence in healthcare research: A path to integrity. [europepmc]
- Artificial Intelligence in Nursing: Technological Benefits to Nurse's Mental Health and Patient Care Quality. [europepmc]
- 2024 EACTS/EACTAIC/EBCP Guidelines on cardiopulmonary bypass in adult cardiac surgery. [europepmc]
- 2024 EACTS/EACTAIC/EBCP Guidelines on cardiopulmonary bypass in adult cardiac surgery. [europepmc]
- Bridging the Gap: From AI Success in Clinical Trials to Real-World Healthcare Implementation-A Narrative Review. [europepmc]
- The role of nanomedicine and artificial intelligence in cancer health care: individual applications and emerging integrations-a narrative review. [europepmc]
- Ethical and legal considerations in healthcare AI: innovation and policy for safe and fair use. [europepmc]
Related