NDLI: Learning model-free robot control by a Monte Carlo EM algorithm

Content Provider	Springer Nature Link
Author	Vlassis, Nikos Toussaint, Marc Kontes, Georgios Piperidis, Savas
Copyright Year	2009
Abstract	We address the problem of learning robot control by model-free reinforcement learning (RL). We adopt the probabilistic model for model-free RL of Vlassis and Toussaint (Proceedings of the international conference on machine learning, Montreal, Canada, 2009), and we propose a Monte Carlo EM algorithm (MCEM) for control learning that searches directly in the space of controller parameters using information obtained from randomly generated robot trajectories. MCEM is related to, and generalizes, the PoWER algorithm of Kober and Peters (Proceedings of the neural information processing systems, 2009). In the finite-horizon case MCEM reduces precisely to PoWER, but MCEM can also handle the discounted infinite-horizon case. An interesting result is that the infinite-horizon case can be viewed as a ‘randomized’ version of the finite-horizon case, in the sense that the length of each sampled trajectory is a random draw from an appropriately constructed geometric distribution. We provide some preliminary experiments demonstrating the effects of fixed (PoWER) vs randomized (MCEM) horizon length in two simulated and one real robot control tasks.
Starting Page	123
Ending Page	130
Page Count	8
File Format	PDF
ISSN	09295593
Journal	Autonomous Robots
Volume Number	27
Issue Number	2
e-ISSN	15737527
Language	English
Publisher	Springer US
Publisher Date	2009-08-11
Publisher Place	Boston
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Model-free robot control Reinforcement learning Probabilistic inference EM algorithm Simulation and Modeling Mechanical Engineering Computer Imaging, Vision, Pattern Recognition and Graphics Electrical Engineering Control , Robotics, Mechatronics Artificial Intelligence (incl. Robotics)
Content Type	Text
Resource Type	Article
Subject	Artificial Intelligence

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Reinforcement learning for robot soccer

Learning state representations with robotic priors

DCOB: Action space for reinforcement learning of high DoF robots

Robot Motion Planning

Guest editorial: Special issue on robot learning, Part A

Guest editorial: Special issue on robot learning, Part B

Learning the behavior model of a robot

Learning domain structure through probabilistic policy reuse in reinforcement learning

Nonparametric representation of an approximated Poincaré map for learning biped locomotion

Learning model-free robot control by a Monte Carlo EM algorithm

Similar Documents

Reinforcement learning for robot soccer

Learning state representations with robotic priors

DCOB: Action space for reinforcement learning of high DoF robots

Robot Motion Planning

Guest editorial: Special issue on robot learning, Part A

Guest editorial: Special issue on robot learning, Part B

Learning the behavior model of a robot

Learning domain structure through probabilistic policy reuse in reinforcement learning

Nonparametric representation of an approximated Poincaré map for learning biped locomotion

Learning model-free robot control by a Monte Carlo EM algorithm