NDLI: Experience replay for least-squares policy iteration

Please wait, while we are loading the content...

Year : 2016 Volume : 3

Year : 2015 Volume : 2

Year : 2014 Volume : 1

Issue 4

Issue 3

Contents

Guest editorial for special issue on extensions of reinforcement learning and adaptive control

Off-policy reinforcement learning with Gaussian processes

Concurrent learning-based approximate feedback-Nash equilibrium solution of N-player nonzero-sum differential games

Clique-based cooperative multiagent reinforcement learning using factor graphs

Reinforcement learning transfer based on subgoal discovery and subtask similarity

Closed-loop P-type iterative learning control of uncertain linear distributed parameter systems

Experience replay for least-squares policy iteration

Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems

Robust adaptive model tracking for distributed parameter control of linear infinite-dimensional systems in Hilbert space

Adaptive iterative learning control for a class of nonlinear time-varying systems with unknown delays and input dead-zone

An improved result of multiple model iterative learning control

Continuous action reinforcement learning for control-affine systems with unknown dynamics

The Chinese Association of Automation

Information for authors

Issue 2

Issue 1

Experience replay for least-squares policy iteration

Content Provider	IEEE Xplore Digital Library
Author	Liu, Quan Zhou, Xin Zhu, Fei Fu, Qiming Fu, Yuchen
Copyright Year	2014
Abstract	Policy iteration, which evaluates and improves the control policy iteratively, is a reinforcement learning method. Policy evaluation with the least-squares method can draw more useful information from the empirical data and therefore improve the data validity. However, most existing online least-squares policy iteration methods only use each sample just once, resulting in the low utilization rate. With the goal of improving the utilization efficiency, we propose an experience replay for least-squares policy iteration (ERLSPI) and prove its convergence. ERLSPI method combines online least-squares policy iteration method with experience replay, stores the samples which are generated online, and reuses these samples with least-squares method to update the control policy. We apply the ERLSPI method for the inverted pendulum system, a typical benchmark testing. The experimental results show that the method can effectively take advantage of the previous experience and knowledge, improve the empirical utilization efficiency, and accelerate the convergence speed.
Starting Page	274
Ending Page	281
Page Count	8
File Size	563346
File Format	PDF
ISSN	23299266
Volume Number	1
Issue Number	3
Language	English
Publisher Date	2014-01-01
Access Restriction	One Nation One Subscription (ONOS)
Rights Holder	Chinese Association of Automation
Subject Keyword	Learning (artificial intelligence) Markov processes Computational efficiency Convergence Least squares methods policy iteration reinforcement learning experience replay least-squares
Content Type	Text
Resource Type	Article
Subject	Control and Optimization Artificial Intelligence Control and Systems Engineering Information Systems

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Online least-squares policy iteration for reinforcement learning control

Off-policy reinforcement learning with Gaussian processes

Kernel-Based Least Squares Policy Iteration for Reinforcement Learning

Finite-Sample Analysis of Least-Squares Policy Iteration

Finite-Sample Analysis of Least-Squares Policy Iteration

Stationary Fuzzy Fokker–Planck Learning for Derivative-Free Optimization

Adaptive pinpoint and fuel efficient mars landing using reinforcement learning

Continuous action reinforcement learning for control-affine systems with unknown dynamics

Hybrid least-squares algorithms for approximate policy evaluation

Experience replay for least-squares policy iteration

Similar Documents

Online least-squares policy iteration for reinforcement learning control

Off-policy reinforcement learning with Gaussian processes

Kernel-Based Least Squares Policy Iteration for Reinforcement Learning

Finite-Sample Analysis of Least-Squares Policy Iteration

Finite-Sample Analysis of Least-Squares Policy Iteration

Stationary Fuzzy Fokker–Planck Learning for Derivative-Free Optimization

Adaptive pinpoint and fuel efficient mars landing using reinforcement learning

Continuous action reinforcement learning for control-affine systems with unknown dynamics

Hybrid least-squares algorithms for approximate policy evaluation

Experience replay for least-squares policy iteration