Preprint
ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guarantees
ArXiv.org
29 Jun 2024
Abstract
Uncertainty quantification (UQ) in natural language generation (NLG) tasks remains an open challenge, exacerbated by the closed-source nature of the latest large language models (LLMs). This study investigates applying conformal prediction (CP), which can transform any heuristic uncertainty notion into rigorous prediction sets, to black-box LLMs in open-ended NLG tasks. We introduce a novel uncertainty measure based on self-consistency theory, and then develop a conformal uncertainty criterion by integrating the uncertainty condition aligned with correctness into the CP algorithm. Empirical evaluations indicate that our uncertainty measure outperforms prior state-of-the-art methods. Furthermore, we achieve strict control over the correctness coverage rate utilizing 7 popular LLMs on 4 free-form NLG datasets, spanning general-purpose and medical scenarios. Additionally, the calibrated prediction sets with small size further highlights the efficiency of our method in providing trustworthy guarantees for practical open-ended NLG applications.
Metrics
30 Record Views
Details
- Title
- ConU: Conformal Uncertainty in Large Language Models with Correctness Coverage Guarantees
- Creators
- Zhiyuan Wang - University of Electronic Science and Technology of ChinaJinhao Duan - Drexel UniversityLu Cheng - University of Illinois ChicagoYue Zhang - Drexel UniversityQingni Wang - University of Electronic Science and Technology of ChinaXiaoshuang Shi - University of Electronic Science and Technology of ChinaKaidi Xu - Drexel UniversityHengtao Shen - University of Electronic Science and Technology of ChinaXiaofeng Zhu - University of Electronic Science and Technology of China
- Publication Details
- ArXiv.org
- Number of pages
- 13
- Resource Type
- Preprint
- Language
- English
- Academic Unit
- Computer Science
- Other Identifier
- 991021889464004721