Conference proceeding
Automatic Prompt Generation and Optimization by Leveraging Large Language Models to Enhance Few-Shot Learning in Biomedical Tasks
IEEE International Conference on Big Data (Print), pp 1645-1654
15 Dec 2024
Abstract
Recent advancements in scaling large language models (LLMs) have enhanced various natural language processing (NLP) tasks. However, open-source moderately sized models, such as BERT, are still needed because of the high computational cost and concerns regarding data privacy from the LLMs, especially in the biomedical area. Prompt-based fine-tuning of BERT has demonstrated good performance in a few-shot setting. However, the prompt selection can result in substantial differences in final accuracy. This study introduces a simple yet effective approach that leverages LLMs, such as GPT-4 Turbo, to automatically generate and optimize task-specific prompts for BERT. Our approach includes two steps: automatic prompt generation and optimization. Initially, we design a framework to generate prompts for LLMs to infer a task-specific candidate prompt set. Subsequently, we employ a dialog with a chatbot to optimize the prompt iteratively. We conduct extensive evaluations and analyses on three different types of biomedical benchmarks. Our method demonstrates superior 5-shot learning performance, outperforming manual prompts by a substantial margin in low-resource settings, achieving up to a 7% absolute accuracy improvement. These results highlight that our method is a task-agnostic approach to utilizing LLMs and automatically enhancing performance on relatively small open-source models with limited resources and human effort.
Metrics
4 Record Views
Details
- Title
- Automatic Prompt Generation and Optimization by Leveraging Large Language Models to Enhance Few-Shot Learning in Biomedical Tasks
- Creators
- Yiwen Shi - Drexel UniversityXiaohua Hu - Drexel University
- Publication Details
- IEEE International Conference on Big Data (Print), pp 1645-1654
- Publisher
- IEEE
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Information Science (Informatics)
- Scopus ID
- 2-s2.0-85218014716
- Other Identifier
- 991022019491904721