NDLI: On-Line Adaptation of Exploration in the One-Armed Bandit with Covariates Problem

Content Provider	IEEE Xplore Digital Library
Author	Sykulski, A.M. Adams, N.M. Jennings, N.R.
Copyright Year	2010
Abstract	Many sequential decision making problems require an agent to balance exploration and exploitation to maximise long-term reward. Existing policies that address this tradeoff typically have parameters that are set a priori to control the amount of exploration. In finite-time problems, the optimal values of these parameters are highly dependent on the problem faced. In this paper, we propose adapting the amount of exploration performed on-line, as information is gathered by the agent. To this end we introduce a novel algorithm, e-ADAPT, which has no free parameters. The algorithm adapts as it plays and sequentially chooses whether to explore or exploit, driven by the amount of uncertainty in the system. We provide simulation results for the one armed bandit with covariates problem, which demonstrate the effectiveness of e-ADAPT to correctly control the amount of exploration in finite-time problems and yield rewards that are close to optimally tuned off-line policies. Furthermore, we show that e-ADAPT is robust to a high-dimensional covariate, as well as misspecified models. Finally, we describe how our methods could be extended to other sequential decision making problems, such as dynamic bandit problems with changing reward structures.
Starting Page	459
Ending Page	464
File Size	247208
Page Count	6
File Format	PDF
ISBN	9781424492114
DOI	10.1109/ICMLA.2010.74
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2010-12-12
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Radio frequency Uncertainty one-armed bandit problem Estimation Optimized production technology Games Machine learning Exploration-exploitation tradeoff on-line learning Approximation methods sequential decision making
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

On-line adaptation of exploration in the one-armed bandit with covariates problem.

armed bandit problem. NIPS Workshop on On-line Trading of Exploration and Exploitation- (2008)

Bayesian modeling of human sequential decision-making on the multi-armed bandit problem (2008)

Effects of Response Frequency Constraints on Learning in a Non-Stationary Multi-armed Bandit Task

25th Annual Conference on Learning Theory Analysis of Thompson Sampling for the Multi-armed Bandit Problem

Exploitative and Exploratory Attention in a Four-Armed Bandit Task

The budgeted multi-armed bandit problem (2004)

The Exploration-Exploitation Tradeoff in Sequential Decision Making Problems

G.: Pure exploration for multi-armed bandit problems

On-Line Adaptation of Exploration in the One-Armed Bandit with Covariates Problem

Similar Documents

On-line adaptation of exploration in the one-armed bandit with covariates problem.

armed bandit problem. NIPS Workshop on On-line Trading of Exploration and Exploitation- (2008)

Bayesian modeling of human sequential decision-making on the multi-armed bandit problem (2008)

Effects of Response Frequency Constraints on Learning in a Non-Stationary Multi-armed Bandit Task

25th Annual Conference on Learning Theory Analysis of Thompson Sampling for the Multi-armed Bandit Problem

Exploitative and Exploratory Attention in a Four-Armed Bandit Task

The budgeted multi-armed bandit problem (2004)

The Exploration-Exploitation Tradeoff in Sequential Decision Making Problems

G.: Pure exploration for multi-armed bandit problems

On-Line Adaptation of Exploration in the One-Armed Bandit with Covariates Problem