Journal article
Use of ChatGPT vs. Google web search in anatomic total shoulder arthroplasty and reverse total shoulder arthroplasty: from a patient’s perspective
Seminars in arthroplasty, v 35(2), pp 158-165
01 Jun 2025
Abstract
Artificial intelligence, which encompasses machine learning and natural language processing has made strides in recent years. Among these models, ChatGPT has gained attention for its abilities to allow for interactive responses. Given the rising popularity of ChatGPT, and its ability to understand medical complexity, this study aimed to evaluate ChatGPT’s utility in answering medical questions vs. the most widely used search engine in the United States, Google. The goal of this study is to replicate a patient’s web search to assess the utility of ChatGPT vs. Google regarding the most popular questions asked by patients undergoing a total shoulder replacement and reverse total shoulder replacement.
A Google web search was performed with the following search terms: “total shoulder replacement” and “reverse total shoulder replacement” and the top 10 frequently asked questions (FAQs) for each term were recorded along with the source. ChatGPT was then asked for the top 10 FAQs searched for total shoulder replacement and reverse total shoulder replacement and responses recorded. A Google web search was then completed using the same search terms and the top 10 FAQs, which included a numerical response were recorded. These questions were then placed into ChatGPT. Both the Google web search and ChatGPT responses to the questions were recorded, along with the source that provided the answer.
Eight of the 20 questions were similar when performing a Google Web search and ChatGPT for the search terms “total shoulder replacement.” Ten of the 20 questions were similar when using the search term “reverse shoulder replacement”. The most common subcategory was “evaluation of surgery” for Google (6 of 20) and “technical” for ChatGPT (5 of 20). The most common sources for answers to the top 10 FAQs for Google for both total shoulder arthroplasty and reverse total shoulder arthroplasty were academic sources (9 of 20).
Google and ChatGPT users ask similar types of questions and reveal similar answers. ChatGPT typically offers more detailed answers but is less transparent at providing its source and generates improper sources up to 47% of the time. However, for properly cited references, ChatGPT typically uses credible government or academic sources compared to Google. Since Google is currently more familiar for its users than ChatGPT, it may be easier to use for patients wanting fast, simple answers to their questions, while ChatGPT may be superior for patients who may have multiple questions that want longer answers.
Metrics
1 Record Views
Details
- Title
- Use of ChatGPT vs. Google web search in anatomic total shoulder arthroplasty and reverse total shoulder arthroplasty: from a patient’s perspective
- Creators
- Alexander K. Hahn - University of ConnecticutGregory M. Connors - Drexel UniversityMartinus Megalla - University of ConnecticutZachary T. Grace - University of ConnecticutBrett J. Croen - University of PennsylvaniaMichael R. Mancini - University of ConnecticutMatthew R. LeVasseur - University of ConnecticutCorey R. Dwyer - University of ConnecticutKatherine J. Coyner (Corresponding Author) - University of Connecticut
- Publication Details
- Seminars in arthroplasty, v 35(2), pp 158-165
- Publisher
- Elsevier
- Number of pages
- 8
- Resource Type
- Journal article
- Language
- English
- Academic Unit
- College of Medicine
- Scopus ID
- 2-s2.0-105003978576
- Other Identifier
- 991022197308104721