NDLI: Learning behaviors via human-delivered discrete feedback: modeling implicit feedback strategies to speed up learning

Content Provider	Springer Nature Link
Author	Loftin, Robert Peng, Bei MacGlashan, James Littman, Michael L. Taylor, Matthew E. Huang, Jeff Roberts, David L.
Copyright Year	2015
Abstract	For real-world applications, virtual agents must be able to learn new behaviors from non-technical users. Positive and negative feedback are an intuitive way to train new behaviors, and existing work has presented algorithms for learning from such feedback. That work, however, treats feedback as numeric reward to be maximized, and assumes that all trainers provide feedback in the same way. In this work, we show that users can provide feedback in many different ways, which we describe as “training strategies.” Specifically, users may not always give explicit feedback in response to an action, and may be more likely to provide explicit reward than explicit punishment, or vice versa, such that the lack of feedback itself conveys information about the behavior. We present a probabilistic model of trainer feedback that describes how a trainer chooses to provide explicit reward and/or explicit punishment and, based on this model, develop two novel learning algorithms (SABL and I-SABL) which take trainer strategy into account, and can therefore learn from cases where no feedback is provided. Through online user studies we demonstrate that these algorithms can learn with less feedback than algorithms based on a numerical interpretation of feedback. Furthermore, we conduct an empirical analysis of the training strategies employed by users, and of factors that can affect their choice of strategy.
Starting Page	30
Ending Page	59
Page Count	30
File Format	PDF
ISSN	13872532
Journal	Autonomous Agents and Multi-Agent Systems
Volume Number	30
Issue Number	1
e-ISSN	15737454
Language	English
Publisher	Springer US
Publisher Date	2015-02-13
Publisher Place	New York
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Learning from feedback Reinforcement learning Bayesian inference Interactive learning Machine learning Human–computer interaction Artificial Intelligence (incl. Robotics) Computer Systems Organization and Communication Networks Computing Methodologies Software Engineering/Programming and Operating Systems User Interfaces and Human Computer Interaction
Content Type	Text
Resource Type	Article
Subject	Artificial Intelligence

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Strategies for simulating pedestrian navigation with multiple reinforcement learning agents

Learning and planning in environments with delayed feedback

Sequentially optimal repeated coalition formation under uncertainty

Hierarchical multi-agent reinforcement learning

Continuous strategy replicator dynamics for multi-agent Q-learning

Generalized multiagent learning with performance bound

Efficient bidding strategies for Cliff-Edge problems

Using informative behavior to increase engagement while learning from human reward

Human-inspired computational fairness

Learning behaviors via human-delivered discrete feedback: modeling implicit feedback strategies to speed up learning

Similar Documents

Strategies for simulating pedestrian navigation with multiple reinforcement learning agents

Learning and planning in environments with delayed feedback

Sequentially optimal repeated coalition formation under uncertainty

Hierarchical multi-agent reinforcement learning

Continuous strategy replicator dynamics for multi-agent Q-learning

Generalized multiagent learning with performance bound

Efficient bidding strategies for Cliff-Edge problems

Using informative behavior to increase engagement while learning from human reward

Human-inspired computational fairness

Learning behaviors via human-delivered discrete feedback: modeling implicit feedback strategies to speed up learning