Logo image
Back
Regression Oracles and Exploration Strategies for Short-Horizon Multi-Armed Bandits
Conference proceeding   Open access

Regression Oracles and Exploration Strategies for Short-Horizon Multi-Armed Bandits

Robert C. Gray, Jichen Zhu, Santiago Ontanon and IEEE
2020 IEEE CONFERENCE ON GAMES (IEEE COG 2020), v 2020-
01 Jan 2020
DOI: https://doi.org/10.1109/CoG47356.2020.9231529
url
http://arxiv.org/abs/2102.05263View
Accepted (AM) Open arXiv.org - Non-exclusive license to distribute

Abstract

Computer Science, Artificial Intelligence Computer Science, Software Engineering Science & Technology Computer Science Technology
This paper explores multi-armed bandit (MAB) strategies in very short horizon scenarios, i.e., when the bandit strategy is only allowed very few interactions with the environment. This is an understudied setting in the MAB literature with many applications in the context of games, such as player modeling. Specifically, we pursue three different ideas. First, we explore the use of regression oracles, which replace the simple average used in strategies such as epsilon-greedy with linear regression models. Second, we examine different exploration patterns such as forced exploration phases. Finally, we introduce a new variant of the UCB1 strategy called UCBT that has interesting properties and no tunable parameters. We present experimental results in a domain motivated by exergames, where the goal is to maximize a player's daily steps. Our results show that the combination of epsilon-greedy or epsilon-decreasing with regression oracles outperforms all other tested strategies in the short horizon setting.

Metrics

7 Record Views
5 citations in Web of Science
7 citations in Scopus

Details

InCites Highlights

Data related to this publication, from InCites Benchmarking & Analytics tool:

Collaboration types
Industry collaboration
Domestic collaboration
Web of Science research areas
Computer Science, Artificial Intelligence
Computer Science, Software Engineering
Logo image