CSP Seminar

CANCELLED Simple Regret Minimization for Contextual Bandits

Clay Scott

University of Michigan, Department of Electrical Engineering and Computer Science
Thursday, October 11, 2018
4:00pm - 5:00pm
1005 EECS

Add to Google Calendar

About the Event

****CANCELLED*******************************************************************************************************There are two variants of the classical multiarmed bandit (MAB) problem that have received considerable attention from machine learning researchers in recent years: contextual bandits and simple regret minimization. Contextual bandits are a sub-class of MABs where, at every time step, the learner has access to side information that is predictive of the best arm. Simple regret minimization assumes that the learner only incurs regret after a pure exploration phase. In this work, we study simple regret minimization for contextual bandits. Motivated by applications where the learner has separate training and autonomous modes, we consider the setting where the learner experiences a pure exploration phase, where feedback is received after every action but no regret is incurred, followed by a pure exploitation phase, in which regret is incurred but there is no feedback. We present the Contextual-Gap algorithm and establish performance guarantees on the simple regret, i.e., the regret during the pure exploitation phase. Our experiments examine a novel application to adaptive sensor selection for magnetic field estimation in interplanetary spacecraft, and demonstrate considerable improvement over algorithms designed to minimize the cumulative regret.


Clay Scott received his PhD in Electrical Engineering from Rice University in 2004, and joined the University of Michigan in 2006 with a primary appointment in EECS. His research interests focus on statistical machine learning theory and algorithms, with an emphasis on nonparametric methods for supervised and unsupervised learning. He has also worked on a number of applications stemming from various scientific disciplines, including brain imaging, nuclear threat detection, environmental monitoring, and computational biology. In 2010 he received the Career Award from the National Science Foundation.

Additional Information

Contact: Judi Jones

Phone: 763-8557


Sponsor(s): ECE-Systems

Faculty Sponsor: Vijay Subramanian

Open to: Public