NDLI: Transfer in variable-reward hierarchical reinforcement learning

Content Provider	Springer Nature Link
Author	Mehta, Neville Natarajan, Sriraam Tadepalli, Prasad Fern, Alan
Copyright Year	2008
Abstract	Transfer learning seeks to leverage previously learned tasks to achieve faster learning in a new task. In this paper, we consider transfer learning in the context of related but distinct Reinforcement Learning (RL) problems. In particular, our RL problems are derived from Semi-Markov Decision Processes (SMDPs) that share the same transition dynamics but have different reward functions that are linear in a set of reward features. We formally define the transfer learning problem in the context of RL as learning an efficient algorithm to solve any SMDP drawn from a fixed distribution after experiencing a finite number of them. Furthermore, we introduce an online algorithm to solve this problem, Variable-Reward Reinforcement Learning (VRRL), that compactly stores the optimal value functions for several SMDPs, and uses them to optimally initialize the value function for a new SMDP. We generalize our method to a hierarchical RL setting where the different SMDPs share the same task hierarchy. Our experimental results in a simplified real-time strategy domain show that significant transfer learning occurs in both flat and hierarchical settings. Transfer is especially effective in the hierarchical setting where the overall value functions are decomposed into subtask value functions which are more widely amenable to transfer across different SMDPs.
Starting Page	289
Ending Page	312
Page Count	24
File Format	PDF
ISSN	08856125
Journal	Machine Learning
Volume Number	73
Issue Number	3
e-ISSN	15730565
Language	English
Publisher	Springer US
Publisher Date	2008-06-03
Publisher Place	Boston
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Hierarchical reinforcement learning Transfer learning Average-reward learning Multi-criteria learning Language Translation and Linguistics Simulation and Modeling Computing Methodologies Automation and Robotics Artificial Intelligence (incl. Robotics)
Content Type	Text
Resource Type	Article
Subject	Artificial Intelligence Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Flexible latent variable models for multi-task learning

Integrating Guidance into Relational Reinforcement Learning

Machine learning and games

Guest editorial: Learning theory

Convex multi-task feature learning

A Reinforcement Learning Scheme for a Partially-Observable Multi-Agent Game

Multi-Class Learning by Smoothed Boosting

Guest editor’s introduction: special issue on inductive transfer learning

Graph kernels and Gaussian processes for relational reinforcement learning

Transfer in variable-reward hierarchical reinforcement learning

Similar Documents

Flexible latent variable models for multi-task learning

Integrating Guidance into Relational Reinforcement Learning

Machine learning and games

Guest editorial: Learning theory

Convex multi-task feature learning

A Reinforcement Learning Scheme for a Partially-Observable Multi-Agent Game

Multi-Class Learning by Smoothed Boosting

Guest editor’s introduction: special issue on inductive transfer learning

Graph kernels and Gaussian processes for relational reinforcement learning

Transfer in variable-reward hierarchical reinforcement learning