NDLI: Model learning actor-critic algorithms: Performance evaluation in a motion control task

Content Provider	IEEE Xplore Digital Library
Author	Grondman, I. Busoniu, L. Babuska, R.
Copyright Year	2012
Description	Author affiliation: CNRS, Research Center for Automatic Control, University of Lorraine, Nancy, France (Busoniu, L.) \|\| Delft Center for Systems and Control of Delft University of Technology, The Netherlands (Grondman, I.; Babuska, R.)
Abstract	Reinforcement learning (RL) control provides a means to deal with uncertainty and nonlinearity associated with control tasks in an optimal way. The class of actor-critic RL algorithms proved useful for control systems with continuous state and input variables. In the literature, model-based actor-critic algorithms have recently been introduced to considerably speed up the the learning by constructing online a model through local linear regression (LLR). It has not been analyzed yet whether the speed-up is due to the model learning structure or the LLR approximator. Therefore, in this paper we generalize the model learning actor-critic algorithms to make them suitable for use with an arbitrary function approximator. Furthermore, we present the results of an extensive analysis through numerical simulations of a typical nonlinear motion control problem. The LLR approximator is compared with radial basis functions (RBFs) in terms of the initial convergence rate and in terms of the final performance obtained. The results show that LLR-based actor-critic RL outperforms the RBF counterpart: it gives quick initial learning and comparable or even superior final control performance.
Starting Page	5272
Ending Page	5277
File Size	364787
Page Count	6
File Format	PDF
ISBN	9781467320658
ISSN	07431546
e-ISBN	9781467320665
e-ISBN	9781467320641
DOI	10.1109/CDC.2012.6426427
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2012-12-10
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Approximation algorithms Function approximation Mathematical model Linear regression Tuning Learning
Content Type	Text
Resource Type	Article
Subject	Control and Optimization Control and Systems Engineering Modeling and Simulation

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Efficient Model Learning Methods for Actor–Critic Control

TD-learning with exploration

Least squares temporal difference actor-critic methods with applications to robot motion control

Online support vector regression based actor-critic method

Policy-Gradient Based Actor-Critic Algorithms

OnActor-Critic Algorithms

Actor-Critic--Type Learning Algorithms for Markov Decision Processes

Adaptive bases for Q-learning

Incremental Receptive Field Weighted Actor-Critic

Model learning actor-critic algorithms: Performance evaluation in a motion control task

Similar Documents

Efficient Model Learning Methods for Actor–Critic Control

TD-learning with exploration

Least squares temporal difference actor-critic methods with applications to robot motion control

Online support vector regression based actor-critic method

Policy-Gradient Based Actor-Critic Algorithms

OnActor-Critic Algorithms

Actor-Critic--Type Learning Algorithms for Markov Decision Processes

Adaptive bases for Q-learning

Incremental Receptive Field Weighted Actor-Critic

Model learning actor-critic algorithms: Performance evaluation in a motion control task