Files and links (1)
Preprint (Author's original)arXiv.org - Non-exclusive license to distribute, Open
Preprint
Beyond Self-Consistency: Ensemble Reasoning Boosts Consistency and Accuracy of LLMs in Cancer Staging
19 Apr 2024
Abstract
Advances in large language models (LLMs) have encouraged their adoption in
the healthcare domain where vital clinical information is often contained in
unstructured notes. Cancer staging status is available in clinical reports, but
it requires natural language processing to extract the status from the
unstructured text. With the advance in clinical-oriented LLMs, it is promising
to extract such status without extensive efforts in training the algorithms.
Prompting approaches of the pre-trained LLMs that elicit a model's reasoning
process, such as chain-of-thought, may help to improve the trustworthiness of
the generated responses. Using self-consistency further improves model
performance, but often results in inconsistent generations across the multiple
reasoning paths. In this study, we propose an ensemble reasoning approach with
the aim of improving the consistency of the model generations. Using an open
access clinical large language model to determine the pathologic cancer stage
from real-world pathology reports, we show that the ensemble reasoning approach
is able to improve both the consistency and performance of the LLM in
determining cancer stage, thereby demonstrating the potential to use these
models in clinical or other domains where reliability and trustworthiness are
critical.
Metrics
17 Record Views
Details
- Title
- Beyond Self-Consistency: Ensemble Reasoning Boosts Consistency and Accuracy of LLMs in Cancer Staging
- Creators
- Chia-Hsuan ChangMary M LucasYeawon LeeChristopher C YangGrace Lu-Yao
- Resource Type
- Preprint
- Language
- English
- Academic Unit
- Information Science
- Other Identifier
- 991021868997404721