Logo image
Back
It’s Not Just for Trust: Designing for Emerging Uses of Explainable AI in Clinical Decision-Making
Journal article   Peer reviewed

It’s Not Just for Trust: Designing for Emerging Uses of Explainable AI in Clinical Decision-Making

Katelyn Morrison, Zexuan Li, Minsuk Kim, Chengqi (Malia) Hong, Jidapa Kraisangka, Priscilla Correa-Jaque, Charles Fauvel, Sandeep Sahay, Rebecca R Vanderpool, Allen Everett, …
ACM transactions on computing for healthcare
15 May 2026
DOI: https://doi.org/10.1145/3815175

Abstract

HCI design and evaluation methods Human-centered computing
Explainable AI (XAI) is often viewed as a mechanism to promote the transparency and interpretability of AI recommendations in high-stakes domains, such as healthcare. This has led many studies to focus on designing and evaluating XAI to foster trust, calibrate reliance, enable algorithmic recourse, and support model understanding. However, this limited design scope restricts our understanding of how XAI can be used more broadly to support critical tasks in complex workflows, such as facilitating shared decision-making and supporting communication between stakeholders. Our work aims to address these critical gaps in the design for and understanding of XAI's emerging uses by iteratively prototyping XAI designs for an AI-powered clinical decision-support system with clinical stakeholders. We then created a high-fidelity prototype from those iterative sessions and used it as a design probe to uncover four emerging uses of XAI: collaboratively exploring treatment options, identifying and reflecting on treatment plans, communicating with stakeholders, and supporting health education. We reflect on the implications of designing XAI for emerging uses in healthcare.

Metrics

1 Record Views

Details

Logo image