NDLI: Distributed reinforcement learning in multi-agent networks

Content Provider	IEEE Xplore Digital Library
Author	Kar, S. Moura, J.M.F. Poor, H.V.
Copyright Year	2013
Description	Author affiliation: Dept. of ECE, Carnegie Mellon Univ., Pittsburgh, PA, USA (Kar, S.; Moura, J.M.F.) \|\| Dept. of EE, Princeton Univ., Princeton, NJ, USA (Poor, H.V.)
Abstract	Distributed reinforcement learning algorithms for collaborative multi-agent Markov decision processes (MDPs) are presented and analyzed. The networked setup consists of a collection of agents (learners) which respond differently (depending on their instantaneous one-stage random costs) to a global controlled state and the control actions of a remote controller. With the objective of jointly learning the optimal stationary control policy (in the absence of global state transition and local agent cost statistics) that minimizes network-averaged infinite horizon discounted cost, the paper presents distributed variants of Q-learning of the consensus + innovations type in which each agent sequentially refines its learning parameters by locally processing its instantaneous payoff data and the information received from neighboring agents. Under broad conditions on the multi-agent decision model and mean connectivity of the inter-agent communication network, the proposed distributed algorithms are shown to achieve optimal learning asymptotically, i.e., almost surely (a.s.) each network agent is shown to learn the value function and the optimal stationary control policy of the collaborative MDP asymptotically. Further, convergence rate estimates for the proposed class of distributed learning algorithms are obtained.
Sponsorship	IEEE Signal Process. Soc.
Starting Page	296
Ending Page	299
File Size	388129
Page Count	4
File Format	PDF
e-ISBN	9781467331463
DOI	10.1109/CAMSAP.2013.6714066
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2013-12-15
Publisher Place	France
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Technological innovation Process control Stochastic processes consensus + innovations Approximation methods collaborative network processing Convergence reinforcement learning distributed Q-learning distributed stochastic approximation Collaboration Multi-agent stochastic control Learning (artificial intelligence)
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

${{cal Q} {cal D}}$-Learning: A Collaborative Distributed Strategy for Multi-Agent Reinforcement Learning Through ${rm Consensus} + {rm Innovations}$

On a consistent procedure for distributed recursive nonlinear least-squares estimation

Distributed parameter estimation with exponential family statistics: Asymptotic efficiency

Distributed state estimation in multi-agent networks

QD-learning: a collaborative distributed strategy for multi-agent reinforcement learning through consensus + innovations

QD-learning: A collaborative distributed strategy for multi-agent reinforcement learning through consensus + innovations (2013)

A novel multi-agent Q-learning algorithm in cooperative multi-agent system

Hysteretic q-learning :an algorithm for decentralized reinforcement learning in cooperative multi-agent teams

Agreement for networked multi-agent systems with noisy communications

Distributed reinforcement learning in multi-agent networks

Similar Documents

${{cal Q} {cal D}}$-Learning: A Collaborative Distributed Strategy for Multi-Agent Reinforcement Learning Through ${rm Consensus} + {rm Innovations}$

On a consistent procedure for distributed recursive nonlinear least-squares estimation

Distributed parameter estimation with exponential family statistics: Asymptotic efficiency

Distributed state estimation in multi-agent networks

QD-learning: a collaborative distributed strategy for multi-agent reinforcement learning through consensus + innovations

QD-learning: A collaborative distributed strategy for multi-agent reinforcement learning through consensus + innovations (2013)

A novel multi-agent Q-learning algorithm in cooperative multi-agent system

Hysteretic q-learning :an algorithm for decentralized reinforcement learning in cooperative multi-agent teams

Agreement for networked multi-agent systems with noisy communications

Distributed reinforcement learning in multi-agent networks