NDLI: Efficient Interactive Multiclass Learning from Binary Feedback

Please wait, while we are loading the content...

Efficient Interactive Multiclass Learning from Binary Feedback

Content Provider	ACM Digital Library
Author	Ngo, Hung Forster, Alexander Vien, Ngo Anh Nagi, Jawas Schmidhuber, Jürgen Luciw, Matthew
Copyright Year	2014
Abstract	We introduce a novel algorithm called $\textit{upper}$ $\textit{confidence}-\textit{weighted}$ $\textit{learning}$ (UCWL) for online multiclass learning from binary feedback (e.g., feedback that indicates whether the prediction was right or wrong). UCWL combines the upper confidence bound (UCB) framework with the soft confidence-weighted (SCW) online learning scheme. In UCB, each instance is classified using both score and uncertainty. For a given instance in the sequence, the algorithm might guess its class label primarily to reduce the class uncertainty. This is a form of informed exploration, which enables the performance to improve with lower sample complexity compared to the case without exploration. Combining UCB with SCW leads to the ability to deal well with noisy and nonseparable data, and state-of-the-art performance is achieved without increasing the computational cost. A potential application setting is human-robot interaction (HRI), where the robot is learning to classify some set of inputs while the human teaches it by providing only binary feedback—or sometimes even the wrong answer entirely. Experimental results in the HRI setting and with two benchmark datasets from other settings show that UCWL outperforms other state-of-the-art algorithms in the online binary feedback setting—and $\textit{surprisingly}$ even sometimes outperforms state-of-the-art algorithms that get full feedback (e.g., the true class label), whereas UCWL gets only binary feedback on the same data sequence.
Starting Page	1
Ending Page	25
Page Count	25
File Format	PDF
ISSN	21606455
e-ISSN	21606463
DOI	10.1145/2629631
Volume Number	4
Issue Number	3
Journal	ACM Transactions on Interactive Intelligent Systems (TiiS)
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2014-08-11
Publisher Place	New York
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Human-robot interaction Active learning Contextual multiarmed bandit Convolutional neural networks Deep neural networks Exploration-exploitation Gesture recognition Online learning Partial feedback Upper confidence bound
Content Type	Text
Resource Type	Article
Subject	Artificial Intelligence Human-Computer Interaction

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Upper confidence weighted learning for efficient exploration in multiclass prediction with binary feedback (2013).

Learning to trade off between exploration and exploitation in multiclass bandit prediction (2011)

Learning to trade off between exploration and exploitation in multiclass bandit prediction

Multiclass classification with bandit feedback using adaptive regularization

Multiclass classification with bandit feedback using adaptive regularization (2011)

Logarithmic online regret bounds for undiscounted reinforcement learning (2007)

Pairwise Regression with Upper Confidence Bound for Contextual Bandit with Multiple Actions

Efficient bandit algorithms for online multiclass prediction (2008)

Human-robot cooperation: fast, interactive learning from binary feedback

Efficient Interactive Multiclass Learning from Binary Feedback

Similar Documents

Upper confidence weighted learning for efficient exploration in multiclass prediction with binary feedback (2013).

Learning to trade off between exploration and exploitation in multiclass bandit prediction (2011)

Learning to trade off between exploration and exploitation in multiclass bandit prediction

Multiclass classification with bandit feedback using adaptive regularization

Multiclass classification with bandit feedback using adaptive regularization (2011)

Logarithmic online regret bounds for undiscounted reinforcement learning (2007)

Pairwise Regression with Upper Confidence Bound for Contextual Bandit with Multiple Actions

Efficient bandit algorithms for online multiclass prediction (2008)

Human-robot cooperation: fast, interactive learning from binary feedback

Efficient Interactive Multiclass Learning from Binary Feedback