NDLI: Analyzing and visualizing multiagent rewards in dynamic and stochastic domains

Content Provider	Springer Nature Link
Author	Agogi, Adrian K. Tumer, Kagan
Copyright Year	2008
Abstract	The ability to analyze the effectiveness of agent reward structures is critical to the successful design of multiagent learning algorithms. Though final system performance is the best indicator of the suitability of a given reward structure, it is often preferable to analyze the reward properties that lead to good system behavior (i.e., properties promoting coordination among the agents and providing agents with strong signal to noise ratios). This step is particularly helpful in continuous, dynamic, stochastic domains ill-suited to simple table backup schemes commonly used in TD(λ)/Q-learning where the effectiveness of the reward structure is difficult to distinguish from the effectiveness of the chosen learning algorithm. In this paper, we present a new reward evaluation method that provides a visualization of the tradeoff between the level of coordination among the agents and the difficulty of the learning problem each agent faces. This method is independent of the learning algorithm and is only a function of the problem domain and the agents’ reward structure. We use this reward property visualization method to determine an effective reward without performing extensive simulations. We then test this method in both a static and a dynamic multi-rover learning domain where the agents have continuous state spaces and take noisy actions (e.g., the agents’ movement decisions are not always carried out properly). Our results show that in the more difficult dynamic domain, the reward efficiency visualization method provides a two order of magnitude speedup in selecting good rewards, compared to running a full simulation. In addition, this method facilitates the design and analysis of new rewards tailored to the observational limitations of the domain, providing rewards that combine the best properties of traditional rewards.
Starting Page	320
Ending Page	338
Page Count	19
File Format	PDF
ISSN	13872532
Journal	Autonomous Agents and Multi-Agent Systems
Volume Number	17
Issue Number	2
e-ISSN	15737454
Language	English
Publisher	Springer US
Publisher Date	2008-04-23
Publisher Place	Boston
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Multiagent learning Visualization Reinforcement learning Reward analysis User Interfaces and Human Computer Interaction Software Engineering/Programming and Operating Systems Computing Methodologies Computer Systems Organization and Communication Networks Artificial Intelligence (incl. Robotics)
Content Type	Text
Resource Type	Article
Subject	Artificial Intelligence

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Generalized multiagent learning with performance bound

Opportunities for multiagent systems and multiagent reinforcement learning in traffic control

Exploring selfish reinforcement learning in repeated games with stochastic rewards

Sequentially optimal repeated coalition formation under uncertainty

Using informative behavior to increase engagement while learning from human reward

Coordinating Multiple Agents via Reinforcement Learning

Hierarchical multi-agent reinforcement learning

A multiagent approach to managing air traffic flow

Multiagent learning in the presence of memory-bounded agents

Analyzing and visualizing multiagent rewards in dynamic and stochastic domains

Similar Documents

Generalized multiagent learning with performance bound

Opportunities for multiagent systems and multiagent reinforcement learning in traffic control

Exploring selfish reinforcement learning in repeated games with stochastic rewards

Sequentially optimal repeated coalition formation under uncertainty

Using informative behavior to increase engagement while learning from human reward

Coordinating Multiple Agents via Reinforcement Learning

Hierarchical multi-agent reinforcement learning

A multiagent approach to managing air traffic flow

Multiagent learning in the presence of memory-bounded agents

Analyzing and visualizing multiagent rewards in dynamic and stochastic domains