Logo image
Back
Custom-Tailored Radiology Research via Retrieval-Augmented Generation: A Secure Institutionally Deployed Large Language Model System
Journal article   Open access   Peer reviewed

Custom-Tailored Radiology Research via Retrieval-Augmented Generation: A Secure Institutionally Deployed Large Language Model System

Michael Welsh, Julian Lopez-Rippe, Dana Alkhulaifat, Vahid Khalkhali, Xinmeng Wang, Mario Sinti-Ycochea and Susan Sotardi
Inventions, v 10(4), 55
08 Jul 2025
DOI: https://doi.org/10.3390/inventions10040055
Featured in Collection :   Research Supported by Drexel Libraries' OA Programs
url
https://doi.org/10.3390/inventions10040055View
Published, Version of Record (VoR)Open Access Discount via Drexel Libraries Read and Publish Program 2025CC BY V4.0,  Open

Abstract

radiology large language models retrieval-augmented generation institutional AI semantic search medical research data privacy Artificial Intelligence or Cybernetics Radiology
Large language models (LLMs) show promise in enhancing medical research through domain-specific question answering. However, their clinical application is limited by hallucination risk, limited domain specialization, and privacy concerns. Public LLMs like GPT-4-Consensus pose challenges for use with institutional data, due to the inability to ensure patient data protection. In this work, we present a secure, custom-designed retrieval-augmented generation (RAG) LLM system deployed entirely within our institution and tailored for radiology research. Radiology researchers at our institution evaluated the system against GPT-4-Consensus through a blinded survey assessing factual accuracy (FA), citation relevance (CR), and perceived performance (PP) using 5-point Likert scales. Our system achieved mean ± SD scores of 4.15 ± 0.99 for FA, 3.70 ± 1.17 for CR, and 3.55 ± 1.39 for PP. In comparison, GPT-4-Consensus obtained 4.25 ± 0.72, 3.85 ± 1.23, and 3.90 ± 1.12 for the same metrics, respectively. No statistically significant differences were observed (p = 0.97, 0.65, 0.42), and 50% of participants preferred our system’s output. These results validate that secure, local RAG-based LLMs can match state-of-the-art performance while preserving privacy and adaptability, offering a scalable tool for medical research environments.

Metrics

9 Record Views
1 citations in Web of Science

Details

InCites Highlights

Data related to this publication, from InCites Benchmarking & Analytics tool:

Web of Science research areas
Engineering, Multidisciplinary
Logo image