Solving the explainable AI conundrum by bridging clinicians' needs and developers' goals.

ABSTRACT: Explainable artificial intelligence (XAI) has emerged as a promising solution for addressing the implementation challenges of AI/ML in healthcare. However, little is known about how developers and clinicians interpret XAI and what conflicting goals and requirements they may have. This paper presents the findings of a longitudinal multi-method study involving 112 developers and clinicians co-designing an XAI solution for a clinical decision support system. Our study identifies three key differences between developer and clinician mental models of XAI, including opposing goals (model interpretability vs. clinical plausibility), different sources of truth (data vs. patient), and the role of exploring new vs. exploiting old knowledge. Based on our findings, we propose design solutions that can

SUBMITTER: Bienefeld N 

PROVIDER: S-EPMC10202353 | biostudies-literature | 2023 May

REPOSITORIES: biostudies-literature