NDLI: A survey of point-based POMDP solvers

Content Provider	Springer Nature Link
Author	Shani, Guy Pineau, Joelle Kaplow, Robert
Copyright Year	2012
Abstract	The past decade has seen a significant breakthrough in research on solving partially observable Markov decision processes (POMDPs). Where past solvers could not scale beyond perhaps a dozen states, modern solvers can handle complex domains with many thousands of states. This breakthrough was mainly due to the idea of restricting value function computations to a finite subset of the belief space, permitting only local value updates for this subset. This approach, known as point-based value iteration, avoids the exponential growth of the value function, and is thus applicable for domains with longer horizons, even with relatively large state spaces. Many extensions were suggested to this basic idea, focusing on various aspects of the algorithm—mainly the selection of the belief space subset, and the order of value function updates. In this survey, we walk the reader through the fundamentals of point-based value iteration, explaining the main concepts and ideas. Then, we survey the major extensions to the basic algorithm, discussing their merits. Finally, we include an extensive empirical analysis using well known benchmarks, in order to shed light on the strengths and limitations of the various approaches.
Starting Page	1
Ending Page	51
Page Count	51
File Format	PDF
ISSN	13872532
Journal	Autonomous Agents and Multi-Agent Systems
Volume Number	27
Issue Number	1
e-ISSN	15737454
Language	English
Publisher	Springer US
Publisher Date	2012-06-08
Publisher Place	Boston
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Partially observable Markov decision processes Decision-theoretic planning Reinforcement learning Artificial Intelligence (incl. Robotics) Computer Systems Organization and Communication Networks Computing Methodologies Software Engineering/Programming and Operating Systems User Interfaces and Human Computer Interaction
Content Type	Text
Resource Type	Article
Subject	Artificial Intelligence

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Decision-theoretic planning under uncertainty with information rewards for active cooperative perception

Shaping multi-agent systems with gradient reinforcement learning

Learning and planning in environments with delayed feedback

Solving efficiently Decentralized MDPs with temporal and resource constraints

Planning in stochastic domains for multiple agents with individual continuous resource state-spaces

Potential-based reward shaping for finite horizon online POMDP planning

A Modified Memory-Based Reinforcement Learning Method for Solving POMDP Problems

Approximate planning for bayesian hierarchical reinforcement learning

Planning for multiple measurement channels in a continuous-state POMDP

A survey of point-based POMDP solvers

Similar Documents

Decision-theoretic planning under uncertainty with information rewards for active cooperative perception

Shaping multi-agent systems with gradient reinforcement learning

Learning and planning in environments with delayed feedback

Solving efficiently Decentralized MDPs with temporal and resource constraints

Planning in stochastic domains for multiple agents with individual continuous resource state-spaces

Potential-based reward shaping for finite horizon online POMDP planning

A Modified Memory-Based Reinforcement Learning Method for Solving POMDP Problems

Approximate planning for bayesian hierarchical reinforcement learning

Planning for multiple measurement channels in a continuous-state POMDP

A survey of point-based POMDP solvers