A Design Framework for operationalizing Trustworthy Artificial Intelligence in Healthcare: Requirements, Tradeoffs and Challenges for its Clinical Adoption
2025/04/27 by Moreno-Sánchez, Pedro A., Del Ser, Javier, van Gils, Mark +1 · 1 citation
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences
paper · doi:10.48550/arxiv.2504.19179
Abstract
Artificial Intelligence (AI) holds great promise for transforming healthcare, particularly in disease diagnosis, prognosis, and patient care. The increasing availability of digital medical data, such as images, omics, biosignals, and electronic health records, combined with advances in computing, has enabled AI models to approach expert-level performance. However, widespread clinical adoption remains limited, primarily due to challenges beyond technical performance, including ethical concerns, regulatory barriers, and lack of trust. To address these issues, AI systems must align with the principles of Trustworthy AI (TAI), which emphasize human agency and oversight, algorithmic robustness, privacy and data governance, transparency, bias and discrimination avoidance, and accountability. Yet, the complexity of healthcare processes (e.g., screening, diagnosis, prognosis, and treatment) and the diversity of stakeholders (clinicians, patients, providers, regulators) complicate the integration of TAI principles. To bridge the gap between TAI theory and practical implementation, this paper proposes a design framework to support developers in embedding TAI principles into medical AI systems. Thus, for each stakeholder identified across various healthcare processes, we propose a disease-agnostic collection of requirements that medical AI systems should incorporate to adhere to the principles of TAI. Additionally, we examine the challenges and tradeoffs that may arise when applying these principles in practice. To ground the discussion, we focus on cardiovascular diseases, a field marked by both high prevalence and active AI innovation, and demonstrate how TAI principles have been applied and where key obstacles persist.
Citations
- TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods
- Physical Adversarial Attacks For Camera-based Smart Systems: Current Trends, Categorization, Applications, Research Challenges, and Future Outlook
- Connecting the Dots in Trustworthy Artificial Intelligence: From AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulation
- Accountability in artificial intelligence: what it is and how it works
- Holding AI to Account: Challenges for the Delivery of Trustworthy AI in Healthcare
- Holding AI to Account: Challenges for the Delivery of Trustworthy AI in Healthcare
- Handling missing values in healthcare data: A systematic review of deep learning-based imputation techniques
- Trustworthy Federated Learning via Blockchain
- From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI
- Trustworthy Artificial Intelligence: A Review
- Trustworthy Augmented Intelligence in Health Care
- Data Harmonisation for Information Fusion in Digital Healthcare: A State-of-the-Art Systematic Review, Meta-Analysis and Future Research Directions
- Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methods
- Transparency of deep neural networks for medical image analysis: A review of interpretability methods
- Unraveling the Hidden Environmental Impacts of AI Solutions for Environment
- Identifying Roles, Requirements and Responsibilities in Trustworthy AI Systems
- Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI
- Clinical AI: opacity, accountability, responsibility and liability
- Creating Value with Artificial Intelligence: A Multi-stakeholder Perspective
- Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI
- Practical Guidance for Involving Stakeholders in Health Research
- Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees
- Fairness Through Computationally-Bounded Awareness
- Path-Specific Counterfactual Fairness
- Online Learning with an Unknown Fairness Metric
- Fairness in Criminal Justice Risk Assessments: The State of the Art
- Fairness in Criminal Justice Risk Assessments: The State of the Art
- Algorithmic decision making and the cost of fairness
- Fair prediction with disparate impact: A study of bias in recidivism\n prediction instruments
- Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments
- Equality of Opportunity in Supervised Learning
- Auditing Black-box Models for Indirect Influence
- Certifying and removing disparate impact
- Certifying and Removing Disparate Impact
Cited by
Related