Files and links (2)
Published, Version of Record (VoR)Open Access (License Unspecified), Open
Conference proceeding
ReTA: Recursively Thinking Ahead to Improve the Strategic Reasoning of Large Language Models
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language, v 1, pp 2232-2246
01 Jan 2024
Abstract
Current logical reasoning evaluations of Large Language Models (LLMs) primarily focus on single-turn and static environments, such as arithmetic problems. The crucial problem of multi-turn, strategic reasoning is under-explored. In this work, we analyze the multi-turn strategic reasoning of LLMs through text-driven complete- and incomplete-information gaming, e.g., board games (TicTac-Toe, Connect-4) and poker games (Texas Hold'em Poker). Specifically, we consider two distinct scenarios: 1) Online Racing, featuring multiple LLMs/agents to facilitate direct competition and comparison; 2) Offline Probing, constructing targeted questions with verified ground truth to evaluate LLMs' strategic behaviors. Experimental results demonstrate that existing state-of-the-art LLMs and reasoning schemes are largely ineffective for strategic reasoning tasks. To mitigate these limitations, we propose a simple yet effective Recursively Thinking-Ahead (ReTA) agent, incorporating a recursive prompting mechanism that automatically analyzes the opponents' future moves/actions and assigns reward signals for these situations, to strengthen the strategic reasoning of LLMs. We hope our work could spur further research and exploration in the multi-turn strategic reasoning of LLMs. The code is available at https://github.com/jinhaoduan/ReTA.
Metrics
4 Record Views
Details
- Title
- ReTA: Recursively Thinking Ahead to Improve the Strategic Reasoning of Large Language Models
- Creators
- Jinhao Duan - Drexel UniversityShiqi Wang - The Art InstitutesJames Diffenderfer - Lawrence Livermore National LaboratoryLichao Sun - Lehigh UniversityTianlong Chen - University of North Carolina at Chapel HillBhavya Kailkhura - Lawrence Livermore National LaboratoryKaidi Xu - Drexel University
- Contributors
- K Duh (Editor)H Gomez (Editor)S Bethard (Editor)
- Publication Details
- Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language, v 1, pp 2232-2246
- Conference
- Conference of the North American Chapter of the Association for Computational Linguistics (Mexico City, 16 Jun 2024–21 Jun 2024)
- Publisher
- Assoc Computational Linguistics-Acl
- Number of pages
- 15
- Grant note
- 2319242 / NSF; National Science Foundation (NSF) 23-ERD-030 / LLNL LDRD Program DE- AC52-07NA27344 / U.S. Department of Energy by the Lawrence Livermore National Laboratory; United States Department of Energy (DOE)
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Computer Science
- Web of Science ID
- WOS:001516375900123
- Scopus ID
- 2-s2.0-85200245700
- Other Identifier
- 991022094664804721
InCites Highlights
Data related to this publication, from InCites Benchmarking & Analytics tool:
- Collaboration types
- Domestic collaboration
- International collaboration
- Web of Science research areas
- Computer Science, Artificial Intelligence
- Computer Science, Interdisciplinary Applications
- Linguistics