TY - GEN
T1 - Randomized allocation with dimension reduction in a bandit problem with covariates
AU - Qian, Wei
AU - Yang, Yuhong
PY - 2012
Y1 - 2012
N2 - Multi-armed bandit problem is an important optimization game requiring an exploration-exploitation tradeoff to achieve optimal total reward. We consider a setting where the rewards of bandit machines are associated with covariates, and focus on the approach of nonparametric estimation of the reward functions together with a randomized allocation to balance the exploration and exploitation. To overcome the curse of dimensionality in nonparametric learning, we propose using dimension reduction methods such as sliced inverse regression (SIR) and likelihood acquired directions (LAD) to reduce the dimension of the covariates. To simultaneously achieve variable selection and dimension reduction, we use coordinate-independent sparse estimation (CISE) for the dimension reduction step. Not knowing which individual dimension reduction method is the best, we show that adaptively combining these dimension reduction methods works really well.
AB - Multi-armed bandit problem is an important optimization game requiring an exploration-exploitation tradeoff to achieve optimal total reward. We consider a setting where the rewards of bandit machines are associated with covariates, and focus on the approach of nonparametric estimation of the reward functions together with a randomized allocation to balance the exploration and exploitation. To overcome the curse of dimensionality in nonparametric learning, we propose using dimension reduction methods such as sliced inverse regression (SIR) and likelihood acquired directions (LAD) to reduce the dimension of the covariates. To simultaneously achieve variable selection and dimension reduction, we use coordinate-independent sparse estimation (CISE) for the dimension reduction step. Not knowing which individual dimension reduction method is the best, we show that adaptively combining these dimension reduction methods works really well.
UR - http://www.scopus.com/inward/record.url?scp=84872951754&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84872951754&partnerID=8YFLogxK
U2 - 10.1109/FSKD.2012.6234368
DO - 10.1109/FSKD.2012.6234368
M3 - Conference contribution
AN - SCOPUS:84872951754
SN - 9781467300223
T3 - Proceedings - 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery, FSKD 2012
SP - 1537
EP - 1541
BT - Proceedings - 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery, FSKD 2012
T2 - 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery, FSKD 2012
Y2 - 29 May 2012 through 31 May 2012
ER -