NDLI: Approximate planning for bayesian hierarchical reinforcement learning

Content Provider	Springer Nature Link
Author	Vien, Ngo Anh Ngo, Hung Lee, Sungyoung Chung, TaeChoong
Copyright Year	2014
Abstract	In this paper, we propose to use hierarchical action decomposition to make Bayesian model-based reinforcement learning more efficient and feasible for larger problems. We formulate Bayesian hierarchical reinforcement learning as a partially observable semi-Markov decision process (POSMDP). The main POSMDP task is partitioned into a hierarchy of POSMDP subtasks. Each subtask might consist of only primitive actions or hierarchically call other subtasks’ policies, since the policies of lower-level subtasks are considered as macro actions in higher-level subtasks. A solution for this hierarchical action decomposition is to solve lower-level subtasks first, then higher-level ones. Because each formulated POSMDP has a continuous state space, we sample from a prior belief to build an approximate model for them, then solve by using a recently introduced Monte Carlo Value Iteration with Macro-Actions solver. We name this method Monte Carlo Bayesian Hierarchical Reinforcement Learning. Simulation results show that our algorithm exploiting the action hierarchy performs significantly better than that of flat Bayesian reinforcement learning in terms of both reward, and especially solving time, in at least one order of magnitude.
Starting Page	808
Ending Page	819
Page Count	12
File Format	PDF
ISSN	0924669X
Journal	Applied Intelligence
Volume Number	41
Issue Number	3
e-ISSN	15737497
Language	English
Publisher	Springer US
Publisher Date	2014-07-20
Publisher Place	Boston
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Reinforcement learning Bayesian model-based RL Bayesian reinforcement learning Model-based reinforcement learning Partially observable Markov decision process (POMDP) Partially observable semi-MDP (POSDMP) Artificial Intelligence (incl. Robotics) Mechanical Engineering Manufacturing, Machines, Tools
Content Type	Text
Resource Type	Article
Subject	Artificial Intelligence

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Bayes-adaptive hierarchical MDPs

Monte-Carlo tree search for Bayesian reinforcement learning

Bayesian Nonparametric Methods for Partially-Observable Reinforcement Learning

Building Adaptive Dialogue Systems Via Bayes-Adaptive POMDPs

Toward Generalization of Automated Temporal Abstraction to Partially Observable Reinforcement Learning

Planning in partially-observable switching-mode continuous domains

Recent Advances in Deep Reinforcement Learning Applications for Solving Partially Observable Markov Decision Processes (POMDP) Problems: Part 1—Fundamentals and Applications in Games, Robotics and Natural Language Processing

A survey of point-based POMDP solvers

Labeling Q-learning in hidden state environments

Approximate planning for bayesian hierarchical reinforcement learning

Similar Documents

Bayes-adaptive hierarchical MDPs

Monte-Carlo tree search for Bayesian reinforcement learning

Bayesian Nonparametric Methods for Partially-Observable Reinforcement Learning

Building Adaptive Dialogue Systems Via Bayes-Adaptive POMDPs

Toward Generalization of Automated Temporal Abstraction to Partially Observable Reinforcement Learning

Planning in partially-observable switching-mode continuous domains

Recent Advances in Deep Reinforcement Learning Applications for Solving Partially Observable Markov Decision Processes (POMDP) Problems: Part 1—Fundamentals and Applications in Games, Robotics and Natural Language Processing

A survey of point-based POMDP solvers

Labeling Q-learning in hidden state environments

Approximate planning for bayesian hierarchical reinforcement learning