2018/11/26 by Owen Lahav, Lahav, Owen, Nicholas Mastronarde +3
Computer Science · #Data Stream Mining Techniques #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning in Healthcare
paper · pdf · doi:10.48550/arxiv.1811.10799
openalex publication_date 2018/11/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Recent efforts in Machine Learning (ML) interpretability have focused on\ncreating methods for explaining black-box ML models. However, these methods\nrely on the assumption that simple approximations, such as linear models or\ndecision-trees, are inherently human-interpretable, which has not been\nempirically tested. Additionally, past efforts have focused exclusively on\ncomprehension, neglecting to explore the trust component necessary to convince\nnon-technical experts, such as clinicians, to utilize ML models in practice. In\nthis paper, we posit that reinforcement learning (RL) can be used to learn what\nis interpretable to different users and, consequently, build their trust in ML\nmodels. To validate this idea, we first train a neural network to provide risk\nassessments for heart failure patients. We then design a RL-based clinical\ndecision-support system (DSS) around the neural network model, which can learn\nfrom its interactions with users. We conduct an experiment involving a diverse\nset of clinicians from multiple institutions in three different countries. Our\nresults demonstrate that ML experts cannot accurately predict which system\noutputs will maximize clinicians' confidence in the underlying neural network\nmodel, and suggest additional findings that have broad implications to the\nfuture of research into ML interpretability and the use of ML in medicine.\n