Conference proceeding
Beyond UCT: MAB Exploration Improvements for Monte Carlo Tree Search
2023 IEEE Conference on Games (CoG), pp 1-8
21 Aug 2023
Abstract
Monte Carlo Tree Search (MCTS) employs Multi-Armed Bandit (MAB) techniques to direct the policy for child node selection during tree construction. Typical MCTS implementations have relied on the Upper Confidence Bounds for Trees (UCT) strategy, which leverages a specific variant of the general Upper Confidence Bounds (UCB) approach. The success of such strategies relies heavily on the proper tuning of the UCB C parameter to guide exploration effectively. This paper examines (1) the advantages of per-arm tuning of C, (2) the potential for a parameter-less UCB variant called UCBT to provide opportunities for automatic derivation of effective C values without prior tuning in a strategy called Poly-UCB1, and (3) the application of both of these concepts toward operational tuning of C during MCTS node expansion and tree construction in a strategy called UCB-Multi. We evaluate our approach in three turn-based, adversarial board games.
Metrics
10 Record Views
2 citations in Scopus
Details
- Title
- Beyond UCT: MAB Exploration Improvements for Monte Carlo Tree Search
- Creators
- Robert C. Gray - Drexel UniversityJichen Zhu - IT University of CopenhagenSantiago Ontanon - Google
- Publication Details
- 2023 IEEE Conference on Games (CoG), pp 1-8
- Publisher
- IEEE
- Resource Type
- Conference proceeding
- Language
- English
- Academic Unit
- Computer Science (Computing)
- Scopus ID
- 2-s2.0-85180552290
- Other Identifier
- 991021869009704721