NDLI: Accelerated gradient temporal difference learning algorithms

Content Provider	IEEE Xplore Digital Library
Author	Meyer, D. Degenne, R. Omrane, A. Hao Shen
Copyright Year	2014
Description	Author affiliation: Inst. for Data Process., Tech. Univ. Munchen, Munich, Germany (Meyer, D.; Degenne, R.; Omrane, A.; Hao Shen)
Abstract	In this paper we study Temporal Difference (TD) Learning with linear value function approximation. The classic TD algorithm is known to be unstable with linear function approximation and off-policy learning. Recently developed Gradient TD (GTD) algorithms have addressed this problem successfully. Despite their prominent properties of good scalability and convergence to correct solutions, they inherit the potential weakness of slow convergence as they are a stochastic gradient descent algorithm. Accelerated stochastic gradient descent algorithms have been developed to speed up convergence, while still keeping computational complexity low. In this work, we develop an accelerated stochastic gradient descent method for minimizing the Mean Squared Projected Bellman Error (MSPBE), and derive a bound for the Lipschitz constant of the gradient of the MSPBE, which plays a critical role in our proposed accelerated GTD algorithms. Our comprehensive numerical experiments demonstrate promising performance in solving the policy evaluation problem, in comparison to the GTD]algorithm family. In particular, accelerated TDC surpasses state-of-the-art algorithms.
Starting Page	1
Ending Page	8
File Size	665794
Page Count	8
File Format	PDF
ISBN	9781479945528
DOI	10.1109/ADPRL.2014.7010611
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2014-12-09
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Approximation algorithms Vectors Acceleration Function approximation Convergence Radio access networks
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Diffusion gradient temporal difference for cooperative reinforcement learning with linear function approximation

Convergence accelerated by the improvements of stepsize and gradient in SPSA

Convergence analysis of temporal-difference learning algorithms with linear function approximation

Asymptotic analysis of temporal-difference learning algorithms with linear function approximation

Online dictionary learning from big data using accelerated stochastic approximation algorithms

An exemplar test problem on parameter convergence analysis of temporal difference algorithms

Feature Search in the Grassmanian in Online Reinforcement Learning

Historical Temporal Difference Learning: Some Initial Results

Fast gradient-descent methods for temporal-difference learning with linear function approximation (2009)

Accelerated gradient temporal difference learning algorithms

Similar Documents

Diffusion gradient temporal difference for cooperative reinforcement learning with linear function approximation

Convergence accelerated by the improvements of stepsize and gradient in SPSA

Convergence analysis of temporal-difference learning algorithms with linear function approximation

Asymptotic analysis of temporal-difference learning algorithms with linear function approximation

Online dictionary learning from big data using accelerated stochastic approximation algorithms

An exemplar test problem on parameter convergence analysis of temporal difference algorithms

Feature Search in the Grassmanian in Online Reinforcement Learning

Historical Temporal Difference Learning: Some Initial Results

Fast gradient-descent methods for temporal-difference learning with linear function approximation (2009)

Accelerated gradient temporal difference learning algorithms