Journal article
Investigating the Accuracy and Completeness of an Artificial Intelligence Large Language Model About Uveitis: An Evaluation of ChatGPT
OCULAR IMMUNOLOGY AND INFLAMMATION
18 Feb 2024
PMID: 38394625
Featured in Collection : UN Sustainable Development Goals @ Drexel
Abstract
PurposeTo assess the accuracy and completeness of ChatGPT-generated answers regarding uveitis description, prevention, treatment, and prognosis.MethodsThirty-two uveitis-related questions were generated by a uveitis specialist and inputted into ChatGPT 3.5. Answers were compiled into a survey and were reviewed by five uveitis specialists using standardized Likert scales of accuracy and completeness.ResultsIn total, the median accuracy score for all the uveitis questions (n = 32) was 4.00 (between more correct than incorrect and nearly all correct), and the median completeness score was 2.00 (adequate, addresses all aspects of the question and provides the minimum amount of information required to be considered complete). The interrater variability assessment had a total kappa value of 0.0278 for accuracy and 0.0847 for completeness.ConclusionChatGPT can provide relatively high accuracy responses for various questions related to uveitis; however, the answers it provides are incomplete, with some inaccuracies. Its utility in providing medical information requires further validation and development prior to serving as a source of uveitis information for patients.
Metrics
Details
- Title
- Investigating the Accuracy and Completeness of an Artificial Intelligence Large Language Model About Uveitis: An Evaluation of ChatGPT
- Publication Details
- OCULAR IMMUNOLOGY AND INFLAMMATION
- Publisher
- TAYLOR & FRANCIS INC; PHILADELPHIA
- Resource Type
- Journal article
- Language
- English
- Academic Unit
- Drexel University
- Web of Science ID
- WOS:001168224600001
- Scopus ID
- 2-s2.0-85186468826
- Other Identifier
- 991021861170904721
UN Sustainable Development Goals (SDGs)
This publication has contributed to the advancement of the following goals:
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Collaboration types
- Domestic collaboration
- Web of Science research areas
- Ophthalmology