Journal article
Pitfalls and Best Practices in Evaluation of AI Algorithmic Biases in Radiology
Radiology, v 315(2), e241674
01 May 2025
PMID: 40392092
Abstract
Despite growing awareness of problems with fairness in artificial intelligence (AI) models in radiology, evaluation of algorithmic biases, or AI biases, remains challenging due to various complexities. These include incomplete reporting of demographic information in medical imaging datasets, variability in definitions of demographic categories, and inconsistent statistical definitions of bias. To guide the appropriate evaluation of AI biases in radiology, this article summarizes the pitfalls in the evaluation and measurement of algorithmic biases. These pitfalls span the spectrum from the technical (eg, how different statistical definitions of bias impact conclusions about whether an AI model is biased) to those associated with social context (eg, how different conventions of race and ethnicity impact identification or masking of biases). Actionable best practices and future directions to avoid these pitfalls are summarized across three key areas:
medical imaging datasets,
demographic definitions, and
statistical evaluations of bias. Although AI bias in radiology has been broadly reviewed in the recent literature, this article focuses specifically on underrecognized potential pitfalls related to the three key areas. By providing awareness of these pitfalls along with actionable practices to avoid them, exciting AI technologies can be used in radiology for the good of all people.
Metrics
1 Record Views
Details
- Title
- Pitfalls and Best Practices in Evaluation of AI Algorithmic Biases in Radiology
- Creators
- Paul H Yi - Western UniversityPreetham Bachina - Western UniversityBeepul Bharti - Western UniversitySean P Garin - Western UniversityAdway Kanhere - Western UniversityPranav Kulkarni - Western UniversityDavid Li - Western UniversityVishwa S Parekh - Western UniversitySamantha M Santomartino - Western UniversityLinda Moy - Western UniversityJeremias Sulam - Western University
- Publication Details
- Radiology, v 315(2), e241674
- Publisher
- Radiological Society of North America
- Grant note
- R01 CA287422 / NCI NIH HHS
- Resource Type
- Journal article
- Language
- English
- Academic Unit
- Radiology (Radiologic Sciences)
- Web of Science ID
- WOS:001496455600005
- Scopus ID
- 2-s2.0-105006563636
- Other Identifier
- 991022197411304721